Thermodynamic Irreversibility from high-dimensional Hamiltonian Chaos 



This paper discusses the thermodynamic irreversibility reahzed in high-dimensional 
Hamiltonian systems with a time-dependent parameter. A new quantity, the irreversible 
information loss, is defined from the Lyapunov analysis so as to characterize the thermody- 
namic irreversibility. It is proved that this new quantity satisfies an inequality associated 
with the second law of thermodynamics. Based on the assumption that these systems possess 
the mixing property and certain large deviation properties in the thermodynamic limit, it is 
argued reasonably that the most probable value of the irreversible information loss is equal 
to the change of the Boltzmann entropy in statistical mechanics, and that it is always a 
non-negative value. The consistency of our argument is confirmed by numerical experiments 
with the aid of the definition of a quantity we refer to as the excess information loss. 



Thermodynamics formalizes a fundamental limitation of possible processes between equilibrium 
states. In particular, when a thermodynamic system is enclosed by adiabatic walls, the limitation 
is represented by, for example, a fact that, given a system in some initial state, it is not possible to 
lower the system's energy by first changing some of its other extensive variables and then returning 
them to their original values. Contrastingly, the energy of the system can be increased by the similar 
change of the other extensive variables. These two facts make clear the special nature of energy as 
an extensive variable. This asymmetry is the basis of thermodynamic irreversibility. 

Therrnpdynamics is one of the most elegant theories being based on only a few fundamental 
principles. tl^ However, one may wonder how its principles emerge out of purely mechanical systems. 
Thermodynamic systems consist of many molecules, whose dynamics are described by Hamiltonian 
equations. Thus, in the idealized limit of adiabatic walls, a thermodynamic system can be regarded 
as a Hamiltonian system that is connected to some mechanical apparatus, but does not contact a 
heat reservoir. With this in mind, it may be natural to expect that thermodynamic irreversibility 
can be formalized in Hamiltonian systems. 

Thermodynamic entropy plays a central role in the description of thermodynamic irreversibility, 
and the thermodynamic entropy is generally thought to be given by the logarithm of the number of 
micro-states. This relation, the Boltzmann formula, seems well-established as far as the calculation 
of equilibrium values is concerned. However, it has not yet been shown that the Boltzmann formula 
provides a complete account of irreversibility. & 

In this paper, we discuss thermodynamic irreversibility based on the nature of high-dimensional 
Hamiltonian chaos. As our most notable result, we find a new quantity which satisfies an inequality 
associated with thermodynamic irreversibility. We define this quantity, the "irreversible information 
loss", from dynamical system considerations. Furthermore, we argue reasonably that the irreversible 
information loss is related to the change of the Boltzmann entropy, and this leads us to conclude that 
the Boltzmann entropy does not decrease for any processes in the thermodynamic limit. 
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1.1. Related studies 

This paper provides theoretical arguments for numerical results reported in a previously published 
paper, & and contains a detailed description of the numerical experiment. 

The present study has been carried out under the influence of several related studies. First, the 
attempt to construct^steady state thermodynamics by Oono and Paniconi has provided the direction 
of the present studya^ They have proposed an operational method to obtain non-equilibrium ther- 
modynamic functions. In addition, from a more general viewpoint, they emphasize the importance 
of theory concerning the relation between two different states. Following this manner of thinking, we 
have set out to study thermodynamic irreversibility from dynamical systems. 

The stochastic energetics proposed by Sekimoto has given a nice example of the construction of 
thermodynamics from dynamical systems. Stochastic energetics formalizes energy transformation 
in Langevin dynamics with a clear distinction between heat and work. Sekimoto and Sasa have 
demonstrated the minimum work principle and defined the free energy from this principle. Their 
argument also includes a complementary relation which defines a new thermodynamic function of 
two state variables. Recently, Sekimoto and Oono have constructed an example of steady state 
thermodynamics by analyzing a Langevin dynamical model. lP 

The minimum work principle can be formulated through an equality proposed by Jarzinski.i^ 
This equality may provide a method to find inequalities related to thermodynamics. In fact, Hatano 
has proved a Jarzinski-type equality for the transition between steady states_Lmder a certain condition 
and has derived an inequality related to the steady state thermodynamics. tP 

As discussed by Crooks, 0* the Jaizinski's equality is also related to the fluctuation theorem 
proposed by Evans, Cohen and Morriss.tH' The fluctuation theorem claims a peculiar property of the 
probability of the finite time average of the entropy production in a non-equilibrium steady state. 
Gallovotti and Cohen have presented a mathematical proof of the fluctuation theorem based on the 
assumption that the steady state measure is given by the dynamical measure. E3_Siiice that time, 
it has been shown that the fluctuation theorem holds even in stochastic systems. tJ^'ElP On general 
grounds, Maes has presented an argument that the fluctuation theorem can be understood by a Gibbs 
property of the space-time measure.© 

Transportation coefficients in non-equilibrium steady states have been expressed in terms of 
dynamical system quantities. There are two different approaches for this. In one approach, the 
viscosity is related to the sum of all Lyapunov exponents in a Hamiltonian system supplemented 
with a deterministic thermostatting force. Ej^'llZP The other approach applies to Hamiltonian systems 
with open boundary conditions. Here, the diffusion constant is related to the escape rate which is 
obtained in terms of the difference between the sum of the positive Lyapunov exponents and the 
Kolmogorov- Sinai entropy. IIM^ 

Finally, we mention some recent developments in the understanding of thermodynamics. Lieb and 
Yngvason wrote an important paper on axiomatic thermodynamics. 1^^ They have given an explicit 
expression of the thermodynamic entropy based on a set axioms concerning the adiabatic accessibility, 
and have proven the entropy principle, the second law of thermodynamics. Although their formulation 
is fully mathematical, the idea of the explicit exnression of thermodynamic entropy can be translated 
into standard energetic thermodynamics. ©'©'E^^ 



1.2. Outline of the paper 

This paper consists of nine sections, each of which consists of several subsections. In order 
to give a self-contained explanation, we include a review of thermodynamics, Hamiltonian systems, 
Boltzmann entropy, and Lyapunov analysis. Some of these are no doubt rather well-known topics to 
specialists. However, there are not a large number of people who understand all of them well. Also, it 
was our intention to write this paper so that it can be understood by non-specialists, who have interest 
in the relation between thermodynamic irreversibility and dynamical systems. The organization of 
the paper is summarized below. 

In Section 2 we start with a review of thermodynamics in an adiabatic environment. Thermo- 
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dynamic irreversibility is precisely defined based on basic notions such as state and process. The 
essence of the thermodynamic entropy is described by the entropy principle. We then explain the 
reason why a Hamiltonian system with a time-dependent parameter provides a model for a thermo- 
dynamic system in an adiabatic environment. We assume the microcanonical measure for the initial 
conditions and that the systems possess the mixing property with respect to the measure. We also 
assume the existence of certain large deviation properties in order to establish correspondence with 
the extensivity of thermodynamics. £3* Based on these assumptions, we define the equilibrium state 
and most probable process in Hamiltonian systems. After these preliminaries, we address a main 
question. 

In Section 3 we first review the Boltzmann formula in statistical mechanics. In particular, defining 
the time-dependent Boltzmann entropy, we derive a simple form of the Boltzmann entropy change for 
general processes. Using this formula, we calculate the average of the Boltzmann entropy change for 
a step process, where the average is taken over the initial conditions sampled from the microcanonical 
ensemble on an energy surface. We show that the average value is positive in the thermodynamic 
limit. We further find that the average value is related to the fluctuation of the Boltzmann entropy 
change. 

In Section 4 we review the Lyapunov analysis, which is a standard method to study chaotic 
dynamical systems with numerical experiments. We start with the Gram-Sdimidt decomposition, 
because it is the easiest computational technique for the Lyapunov analysis. c2P We then discuss the 
convergence property of an orthogonal frame. Since the orthogonal frame obtained from convergence 
does not satisfy the transitive property, we define Lyapunov vectors from the orthogonal frame so that 
this property is satisfied. ciP Based on these Lyapunov vectors, we define Lyapunov exponents, local 
expansion ratios, and the information loss rate. In order to recover the symmetry of unstable and 
stable directions, we also define contraction ratios. We then prove a relation between the expansion 
and contraction ratios. We also derive an expression for the weight on trajectory segments. 

In Section 5 we discuss the reversibility of Hamiltonian systems. We relate the evolution map, 
Lyapunov vectors, and local expansion rates for time-reversed systems to those for the original system. 
The reversibility leads to a reversibility paradox. CJ^ In order to resolve the paradox, we need to 
consider the measure for a set of the initial conditions for time-reversed systems. This consideration 
leads to a reversibility relation expressed in terms of probability. 

In Section 6 we begin with the definition of irreversible information loss. Using the reversibility 
relation mentioned above, we prove that the irreversible information loss averaged over the initial 
conditions is always non-negative. We define the most probable value of the irreversible information 
loss in the thermodynamic limit, and we present an argument that this most probable value is equal 
to the Boltzmann entropy change. 

In Section 7 we define a quantity we call excess information loss, because this quantity is more 
tractable than the irreversible information loss. We present a relation between the Boltzmann entropy 
change and the excess information loss based on the assumption that the reversible part of the 
excess information loss is equal to the quasi-static excess information loss. This relation is identical 
to an equality proposed in a previous paper, a) Furthermore, we briefiy discuss a minimum excess 
information loss principle, which may be analogous in some sense to the minimum work principle 
in thermodynamics with an isothermal environment. We also explain the origin of the quasi-static 
excess information loss using Lyapunov analysis. 

In Section 8 we report results of numerical experiments on a Fermi-Pasta-Ulam modeled with a 
time-dependent nonlinear term. With these, we numerically check the assumptions in the arguments 
given in the previous sections, and numerically demonstrate several properties of certain quantities 
such as the Lyapunov exponents and Boltzmann entropy changes in this model. As the main nu- 
merical experiment, we confirm the relation between the Boltzmann entropy change and the excess 
information loss. 

The final section is devoted to concluding remark. 
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1.3. Remarks 

Our theoretical arguments include some non-rigorous, but intuitively reasonable statements. To 
as great an extent as possible, we state explicitly when the assertions we make are assumptions. There 
is one exception, however. We often use the expression o{N) to represent a quantity of negligible 
magnitude compared to N in the limit N ^ oo. This constitutes an order estimate valid in the case 
that the system satisfies an appropriate condition. However, we do not discuss what this condition 
is, nor do we explicitly state that an assumption is involved when we neglect such a quantity. We 
simply expect that the condition is satisfied unless an abnormal situation occurs. 

We use the same font for numbers and vectors. We believe that the difference can be understood 
in the context. Also, a matrix is expressed as A, and Aij denotes the (ij)-element of this matrix. 



§2. Preliminary 



In this section, we review thermodynamics and Hamiltonian systems. We clarify basic assump- 
tions of our theory and address the main question of this paper. 

2.1. Thermodynamic irreversibility 

A thermodynamic system is characterized by the internal energy U and a set of work variables 
{Xj}. When the value of Xi is changed externally, the energy change is induced. The infinitely small 
response dU is written as 

dU = J2^idXi. (2-1) 

i 

In thermodynamics, Xi is chosen as an extensive or intensive variable. Since the internal energy U 
is an extensive variable, Yi is an intensive or extensive variable, respectively. The relation Eq. ( |2T| ) 
is valid for the case that the system is enclosed by adiabatic walls. More formally, Eq. (2-1) should 



be regarded as a mathematical expression of a physical situation that the system is placed in an 
adiabatic environment. 

The equilibrium state U is assumed to be realized when the system is left for a sufficiently long 
time after values of the work variables are fixed. This assumption provides the operational definition 
of the equilibrium state. Also, the equilibrium state U is assumed to be determined uniquely by a 
set of the values of {U,{Xi}). That is, the state E is identified with {U,{Xi}). When the value of 
Xi is changed externally, a transition from an equilibrium state Eq to another one Ei occurs. This 
transition, which is denoted by Eq ^ Ei, is called the thermodynamic process or simply the process. 
More precisely, the process is called the adiabatic process realized in an adiabatic environment. 
However, in the argument below, we use the term process instead of adiabatic process. 

Let Eq and Ei be arbitrary equilibrium states. We then ask whether or not processes Eq —> Ei 
and El Eq are realizable. When both the processes are realizable, these are called reversible 
processes. When only one process Eq Ei is realizable, this process is called the irreversible process. 
We can easily see that the process 

{U,{X^})^{U',{X.^) (2-2) 

provides an example of irreversible processes when U' > U. 

The essence of the thermodynamic entropy is described by the entropy principle: 1^1^ 
There exists an extensive variable S given by a function of E such that the inequality 

S{Ei) > S{Eq) (2-3) 

is satisfied if and only if a process Eq A Ei is realizable. The extensive variable S is determined 
uniquely up to multiplicative and additive arbitrary constants. 

Lieb and Yngvason have proved the entropy principle based on the axioms concerning the adi- 
abatic accessibility. cP Also, in conventional thermodyimniiiis based on work and heat, the entropy 
principle can be proved on some physical assumptions. c3)'ElP 
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2.2. Hamiltonian systems 

A Hamiltonian system is characterized by a Hamiltonian function H{r), where F is a set of 
canonical coordinates {qi} and momentums {pi} 

ri = qi, (2-4) 

rN+i = Pi, (2-5) 

where 1 < i < N . Equations of motion for q^ and pi are given by 

dqi dH 

1F = W.' ^ 



These equations are formally written as 



^ = -— (2-7) 
dt dqi 



dT _ dH 

dt ~ ^ or' ^ 



(2-12) 



where ^7 is a 2N x 2N anti-symmetric matrix which satisfies 

J^ = -l. (2-9) 

Under an initial condition r{0) given at t = 0, the phase space point at time t, r{t), is determined 
by the equation of motion. 

In this paper, we are concerned with Hamiltonian systems with a time-dependent parameter a. 
The energy of the system E at time t is given by 

E{t) = H{r{t),a{t)). (2-10) 

We then obtain the equality 

dE dH dqi dH dpi dH da 

'dt^fr'}~d^i^^Wi^ ~dalil' ^' ' 

dH da 
da dt ' 

where we have used the equations of motion. This equality is rewritten as 

dE = Ada, (2-13) 

where 

'-^) 

Comparing Eq. ( |2T3| ) with Eq. (2T), we find that E and a correspond to the internal energy U 
and a work variable X. This suggests that a Hamiltonian system with a time-depending parameter 
can be a dynamical system model for thermodynamics in an adiabatic environment. We proceed 
to our discussion based on this expectation and attempt to find necessary issues so as to establish 
consistency with thermodynamics. 

Since we are particularly interested in thermodynamic processes, we assume that the value of a 
is changed in a finite time interval [r^, tj], that is, 

^ = (2.15) 

dt ^ ' 

when t ^ In the argument below, we assume the condition 

0<Ti<r/ (2-16) 



without an explicit remark. Note that ^ in Eq. ( p-16|) has been assumed for a technical reason. We 
also represent the protocol of the parameter change by a(). 
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2.3. Measure 

We assume that the initial condition given at t = is sampled from the microcanonical ensemble 
on an energy surface E. The measure for the ensemble is given by the microcanonical measure 

li,Udr; U) = J^J^^^da, (2-17) 

where da is the Lebesgue measure on the energy surface, and \U\ is given by 

Atmc(-^; E) and \S\ are rewritten as 

finAdr; r) = ^^dr6{H{r) - e), (2-19) 
= j dr5{H{r) - E), (2-20) 

where dF is the 2A^-dimensional Lebesgue volume element in the phase space. We also notice that 
Eq. (2-19) is given by 

fl^ciMn, ^) = 1™ „ ^y^^;:p^ (2-21) 

where U o SE denotes a union of energy surfaces from E to E + 6E, A^{r) is a region with a size e 
which includes a point F, and /xl is the 2N dimensional Lebesgue measure. 

We also assume that the systems in question possess are ergodic and process the mixing property. 
Here, a system is called ergodic with respect to the microcanonical measure, when the equality 

hm ^ r dtf{F{t)) = [ fiUdF;U)f{F) (2-22) 

holds for an arbitrary measurable function / and almost all initial conditions F{0) with respect to the 
measure. The mixing property with respect to the microcanonical measure means that the equality 

^hm J i_i^,{dF{Oy,S)f{F{0))g{F{t)) = J fi^,{dF; S)f{F) J i^^,{dF; U)g{F) (2-23) 

holds for arbitrary measurable functions / and g. It is easily proved that a mixing system possesses 
the ergodicity. 

Suppose that the initial condition at time t = —t', t' > 0, is sampled from an ensemble with the 
measure 

lif{dF{-t')- S) = is^,{dF{-t'); U)f{F{-t')), (2-24) 
where / is a measurable function normalized in such a way that 

/ fi^c{dF{-t'y,U)f{F{-t')) = l. (2-25) 

Then, the mixing property leads to 

hm / fif{dF{-t'y,U)g{F{0)) = f f,^,{dF; U)g{F). (2-26) 

That is, the average of g{F{{))) with respect to fif{dF{—t'); U) is the same as the average of g{F{0)) 
with respect to fj,rnc{dF{0); U) when t' — > oo. Using Eq. (l2-26|) , we check numerically the validity of 
the mixing property and we can prepare the microcanonical ensemble at t = in the following way. 

First, we prepare a set of the initial conditions at t = —t' sampled from an ensemble with a 
measure absolutely continuous with respect to the Lebesgue measure on the energy surface U. (It 



Thermodynamic irreversibility from Hamiltonian Chaos 



7 



can be done easily in numerical experiments.) Then, we take an average of a dynamical variable, for 
example A, at t = 0. We carry out two experiments for two different measures assumed at t = —t'. 
If the average values coincide, we may regard that the system possesses the mixing property. □ Also, 
from Eq. ( |2-26D , we find that this average value is the average for the micro canonical measure at 
t = 0, fJ-mcidr; X!). This implies that we can prepare the microcanonical ensemble at t — 0. 

2.4. Thermodynamic limit 

In thermodynamics, the internal energy U is an extensive variable, and a work variable is an 
extensive or intensive variable. In ordeiito establish the consistency with thermodynamics, we assume 
the following large deviation propertyE3) which may be closely related to the extensivity of the energy: 

Let nE{Ei)dE be a probability that final energy after the parameter change takes a value in the 
region [Ei, Ei + dE]. Then, He is written in the form 

nE{Ei) ~ eM-NMEi/N)), (2-27) 

in the appropriate asymptotic limit including N ^ oo. 

Several remarks are mentioned, (i) The appropriate limit in Eq. ( 2-271 ) is called the thermo- 



dynamic limit. In the argument below, the limit — > oo always implies the thermodynamic limit 
without an explicit remark, (ii) The probability of final energy is induced from the measure for the 
ensemble of the initial conditions, (iii) (pE is called a rate function, and is a non-negative convex 
function with zero. The zero of (pE, Ei^, is called the most probable value of Ei/N. 

We next discuss the extensivity or intensivity of a. We pay attention to the case that a is an 
intensive parameter. (The reversed case is similarly discussed.) The variable A then turns out to be 
an extensive variable, which is characterized by the large deviation property: 

Let nji{A')dA be a probability that A takes a value in the region [A', A' + dA] at t = 0. Then, 

is written in the form 

nA{A') ~ expi-N^A'/N)), (2-28) 

in the thermodynamic limit. 

Note that the probability density 77^ is determined by the measure for the ensemble. Since the 
most probable value of A/N, A^^, exists for each energy surface, we write A^{X!) when we emphasize 
the state dependence. 

2.5. Equilibrium state 

We assume that the equilibrium state in thermodynamics corresponds uniquely to the energy 
surface. That is the reason why we used the same symbol U of an energy surface in Section 2.3 as the 
equilibrium state in Section 2.1. Also, the energy surface is specified by a set of quantities {E, a). In 
the argument below, U denotes an equilibrium state, an energy surface and a set of quantities {E, a). 

Let us discuss a condition under which we can know whether or not the equilibrium state is 
realized. The term 'equilibrium' implies that nothing changes due to the balance. Thus, it is natural 
to find a quantity which does not change at equilibrium. Although the energy E does not change 
when t > Tf, it is strange that the equilibrium state is realized immediately after the parameter 
change is finished. The energy cannot be used as an indicator of the equilibrium state. The next 
candidate of the indicator may be the variable A. However, since the trajectory never converges to a 
fixed point, the value of A remains time-dependent. We then notice here that the argument for the 
nature of equilibrium should be developed with the thermodynamic limit. 

Suppose that r{Tf) £ Ui. In general, A{Tf)/N is not equal to A^{IJi). However, from large 
deviation and mixing properties, we can expect 

lim4r^^*(^i) (2-29) 



* However, precisely speaking, this is nothing but a confirmation of one of necessary conditions for the mixing 
property. 
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in the thermodynamic limit. When A{t)/N is sufficiently close to An:{X!i) up to a certain time, □ 
we assume that the state is at the equilibrium. There may be other important physical quantities 
to be checked. However, since we do not have any criteria for the importance, we assume that the 
relaxation of the variable A is enough to identify the equilibrium state. 

2.6. Most probable process 

Suppose that an equilibrium state Uq is realized at t = and that another equilibrium state is 
realized in an energy surface Ui after a sufficiently long time from t = Tf. We call this transition 
a process in the similar way as thermodynamics. However, since Ui depends on r{0), Ui is not 
determined uniquely when we assume the initial energy surface X!q and the protocol of the parameter 
change q(). Here, in order to establish the correspondence with thermodynamics, we assume that 
the large deviation property of the path of E: 

Let He 

:path({-^'(0; ^ t' ^ T})Y\t ^^i^) t>e a path probability that E{t) takes a value in the 
region [E' , E' + dE{t)] at time t. Then, IlE-path is written in the form 

nE:p.thi{E'it),0< t' < r}) ~ exp{-NcPE:p.th{{E\t)/N,0< t' < r}), (2-30) 

in the thermodynamic limit. 

The probability density -/T^ipath is determined by the measure for the ensemble of the initial 
conditions. The rate function (pE-.path is a function of path segments {E'[t)/N,{) < t' < r}, and 
there is a most probable path {E^,{t'),0 < t' < t} which minimizes the rate function. Then, 
since the parameter is changed in a deterministic way, the most probable process is defined as 
{{NE^{t'),a{t')), < t' < Tf}. The most probable process is denoted by 

i^o^*^!, (2-31) 
where Ui = {NE^{Tf),a{Tf)), and it is identified with the thermodynamic process Uq ^i- 

2.7. Main question 

Let us summarize our basic assumptions and address the main question. When one attempts to 
study thermodynamic irreversibility in Hamiltonian systems, there seem three problems. The first 
problem is related to the measure for the initial conditions, where, as one example, a condition which 
determines the most natural measure is concerning. The second problem is related to the reason why 
macroscopic variables behave in a deterministic way. The discussion of large deviation properties is 
one way to consider the problem. 

In this paper, we do not enter these problems deeply. As mentioned above, we assume that 
Hamiltonian systems in question possess the mixing property with respect to the microcanonical 
measure, and we also assume the large deviation properties of A and E in the thermodynamic limit. 
Nevertheless, we believe that there is still an important problem to be solved. We ask how the 
thermodynamic law is established. In other word, we ask whether or not we can find a state variable 
satisfying the entropy principle from dynamical systems. Let us write the question explicitly. 

Let Z'o^^Z'i be an arbitrary most probable process. Then, find a state variable S such that 

S{Ui) > S{Uo), (2-32) 
where the equality holds only when the reversed process Z'l^^Z'o is realizable. 

§3. Statistical mechanics 

In statistical mechanics, the thermodynamic entropy is calculated as the Boltzmann entropy. We 
then review fundamental properties of the Boltzmann entropy and discuss whether or not we can 
answer to the main question by using the Boltzmann entropy. 

* Formally, we should consider the limit N ^ oo before the limit t ^ oo. In the experiment with finite A'^, we 
should focus on an asymptotic regime up to a certain time. 
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The thermodynamic entropy takes a constant value along an arbitrary quasi-static process Uq 
El, which is realized by infinitely slow change of the pa rameter value. Then, in developing statistical 
mechanics, we first attempt to find such a quantity. tZP The adiabatic theorem ensures the existence 
of an invariant quantity along quasi-static processes. We thus start with the adiabatic theorem. 

3.1. Adiabatic theorem 

Let f2(U) be the phase space volume enclosed by an energy surface S = (E, a). 



0{E,a) = J dre{E - H{r,a)). (3-1) 



When the value of a is changed in time, the energy of the system changes. We define the time 
evolution of the phase space volume as 

f2{t) = [2{E{t),a{t)), (3-2) 
where note that E{t) depends on r{0). We then obtain 



dn 



'da^'dE 



da 



(3-3) 



where we have used the equality 

dE da , , , 

IT = d< • <^"" 

which is given by Eq. (2T2). 

By using the expression Eq.( |3T| ), we derive 

IIh-I. (3.5) 
|^ = -|^I(A>,. (3.6) 



where \S\ and {f) ^ are defined as 



\U\ = J dr5{E - H{r,a)), (3-7) 
if)i^ = ^ / jJ^m- (3-8) 



if)^ corresponds to the average of / over the micro-canonical ensemble on the energy surface U. 
Substituting Eqs. (3-5) and (3-6) into Eq. (p-3D, we obtain 



= m^A'J^, (3.10) 

where we have defined a new variable 6 A as 

6A = A-{A)^. (3-11) 

We now prove the adiabatic theorem which states that the equality 

Q{Uo) = f2{Si) (3-12) 

holds for an arbitrary quasi-static process Sq ^ Si. 
(proof) 
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Put = (£"0, ao) and Si = [E^, o-oo)- We decompose the quasi-static process into n quasi-static 
processes such that 

("i'^i) ^ ("i+i'-^i+i)' (3-13) 
where < j < n — 1, and aj+i = aj + Aa with 

Aa=^^^^. (3-14) 
n 

Note that an = aoo and En = E^o- We first assume 

Af2j = f2{Ej+i,aj+i) - Q{Ej,aj), (3-15) 

= OiiAaf) (3-16) 

for large n. We then obtain 

n-l 



n{Eo, ao) - Q{E^,aoo) = ^hm^ ^ 0{{Aaf), (3-17) 



+00 

= hm Oi-), (3-18) 

n^oo 77, 

= 0. (3-19) 

This shows the adiabatic theorem. 

We next prove Eq. (3-16). Without loss of generality, we can assume that the value of a is 
monotonically changed from aj to aj + Aa. Defining the protocol of the parameter change as 

arit) = aj + {Aa)^, (3-20) 
we calculate AQj from Eq. (3-10) in the following way: 



AQj=\mi^j dt\U{t)\5A{t)-^, (3-21) 
= lim r dt\E{t)\5A{t). (3-22) 



When Aa is sufficiently small, AQj is evaluated as 

{Aa 



AQj = \U{0)\ lim r dt6A(t) + 0({Aaf), (3-23) 

T^oo T Jo 

= OiiAaf), (3-24) 



where we have used 



hm - r dtSA(t) = 0, (3-25) 

r^oo r Jo 

which is equivalent to 

}}^^l'dtMt) = {A)^. (3-26) 

This equality holds for almost all initial conditions with respect to the Lebesgue measure on the 
energy surface because of the ergodicity with respect to the microcanonical measure. 

(q.e.d) 
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3.2. Boltzmann entropy 

We define the Boltzmann entropy S^q as 



SB{S) = \ogQ{E), 



(3-27) 



where the Boltzmann constant is assumed to be the unity. For later convenience, we define the 
temperature T{I!) as 



T{E) 



OSb 
dE 



(3-28) 



Although Eq. ( 3-27| ) is the formula which makes us possible to calculate the thermodynamic entropy 
for the equilibrium state U, we define the time evolution of S-q as 



SB{t) = SB{m)- 



Then, from Eqs. (3-10), ( |3^ and (|3^ , we obtain 

dS^ 1 



dt 



The integration of this equation during the time interval [0, r] leads to 

5A{t) da 



dt 



T{U{t)) dt 



(3-29) 



(3-30) 



(3-31) 



Here, we notice that Ada equals to the energy change AE during a time interval [t, t + dt] and 
that (A) J. da may be interpreted as the quasi-static work, Wqs, calculated under the condition that 
the system stays virtually in the energy surface. Wqs is identical to the work done in the actual process 
which can be realized when the system contacts a heat bath with slowly changing temperature. The 
quasi-static heat Qqs from the heat bath is then given by 



Q. 



qs 



AE-W, 



qs 



6 Ada. 



Using Qqs, we rewrite Eq. ( |3-31 ) as 



ASb 



dt 



}_dQ_ 
T dt 

dQqs 



qs 



This should be compared with the Clausius's formula 

AS-- 



d'Q 
T ' 



(3-32) 

(3-33) 
(3-34) 

(3-35) 



where d'Q is an infinitely small quasi-static heat exported from a heat bath. In this way, the Boltz- 
mann entropy turns out to be identified to the thermodynamic entropy. 

3.3. Entropy change for step processes 
We discuss the step process given by 



da 
IE 



6{t)Aa. 



(3-36) 



* See page 37 1 in the Boltzmann's bookcj-' as an explicit presentation of the Boltzmann formula. However, the 
expression of Eq. 



i-27 ) was first proposed by Gibbs as the correspondence of thermodynamic entropy. See page 128-^1 



the Gibbs's book.lfHTThe monograph by P. and T. Ehrenfest is also useful to know contemporary ideas-with them. Li 



Do not confuse it with the time evolution of the //-function in the //-theorem of Boltzmann 
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In the argument below, Sq = [EQ^ao) and Ei = {Ei,ai) denote the initial and final states, respec- 
tively. By substituting Eq. ( F36|) into Eq. (|3^ ), we have 



M(0+) ^ M(0- 



Aa, 



(3-37) 



where note that 6 A and T are discontinuous at t = 0. We consider the average over initial conditions 
sampled from the microcanonical ensemble on the energy surface X!q. This average is denoted by ( )q. 
We calculate (S'b)o as 



+ 0((Z\«)2), 



where Tq = T{X!q). (The proof will be given below.) Further, since the evaluation 

d 



dE 



{5Af)^ = o{N) 



is expected when — > oo, we obtain 



where we have used 



(^Sb)(, = + o(JV, {Aaf ) > 0. 



EdE T ^N' 



(3-38) 



(3-39) 



(3-40) 



(3-41) 



In the thermodynamic limit, {ASb)q is equal to the entropy difference Sb{Ei) — Sb{Eq) for the 
most probable process Eq^^^Ei. Thus, we conclude 



SBiEi)>SBiEo)+oiN, (Aaf) 

for the most probable step process i^o— 

Furthermore, from Eq. ( 3-37 ), the fluctuation ((Z\5b)^)q is calculated as 



(3-42) 



(ASb)' 



rp2 

^0 



{Aaf + o{{Aa) 



Combing this result with Eq. ( 3-40| ), we obtain the equality 



{ASb), = -{{ASBY)^ + o{{Aa)\N). 

This is the fluctuation-response relation for the entropy change. 
Now, we prove Eq. ( ^-38 ). 
(proof) 
We first find 



(3-43) 



(3-44) 



M(0+) ^ M(0_) 



(^^(0+))o 



From Eq. ( p-37 ), we have 



L T{E^) T{Eo) 



+ 0{{Aay^ 



{^Sb)o 



2Tn 



{Aa) + 0{{Aa) 



Let us evaluate {5A{0^))q up to the order of Aa, where 

(M(0+))„ 



(f(no.),a(0,)) 



( ( 5a ) 



(3-45) 
(3-46) 

(3-47) 
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Since -r(0+) = /^(0_), we expand the first term of the right-hand side in such a way that 



— (r(0+),a(0+)) = — (r(0_),a(0_)) + ^(r(0_),a(0_))Z\a + O((Z\a)2) 



Taking the average over the initial conditions, we obtain 



\ da 



+ 



\ da / Q \ da^ 



Aa + OiiAa) 



(3-48) 



(3-49) 



We next evaluate the second term of right-hand side of Eq. ( |3-47 ). 



(3-50) 



We notice that there are four terms which include Aa. (i) Aa appears in l/li^il, (ii) it appears in 
dH/da in the integrand, (iii) it appears in H{r, ai) in the Dirac's delta function, and (iv) it appears 
in El in the Dirac's delta function. We extract the contribution proportional to Aa in each term, 
(i) The first term: The contribution is 



1 d\S\ 



|i7P da 



/a IT 
dr—6iHir,ao)-Eo), 



where we have defined 



We here note that the equality 



_d__ d_ ^_d_ 
da da dE 



dQ_ 

da 



0. 



holds owing to the adiabatic theorem. This equality and the relation 17 = \^\T leads to 



d\S\ 



da 



^ol dT 



Thus, Eq. ( 3-5l| ) becomes 



1 dT 
To da 



Tq da 



(3-51) 

(3-52) 
(3-53) 

(3-54) 



(3-55) 



(2) The second term: Without any calculation, the contribution from the second term is 

ld''H\ 



do? 



(3-56) 



(3) The third and forth terms: In deriving the third and forth terms, we employ the following 
formula 



da 



d 
dE' 

d 
dE' 



fin 



H=E' \^rH\ 
{f)iE',a)\^iE',a 



E'=E 



E'=E 



(3-57) 
(3-58) 
(3-59) 
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Owing to this formula, we calculate the contribution from the third term 

2 





J. d_ 

d I (dH\ 
"dE \ V9a j 



V da 

2 



E 



1 d\u\ 



dE 



( ( 5a ) 

Similarly, the contribution from the forth term is obtained as 



_J d_ 

Ww'dE 
'dH\ d 



\ da / \ o\ da / q' 



/dH\ d l/dHV 
['da/o'dE [\'da/ 



, 1 d\U\ 
l^lo dE 



/dHV^ 
o\da / 



The contributions from the third and forth terms are combined in the form 



1 dU 
'dE 



(^^)^>o-^((^^)^o 



d{A) 



E 



dE 



Then, all the contributions given by Eqs. (l3^ ), (|]5|) and (|3^ are summarized as 



ld_H\ 



/d_HX 



,\da/jjj^ \da/ 
1 dE 



+ 



d^H 
do? 



Aa 



d 



i:dE\^'^n.^-8E{^'^n.^-^ 



where we have used the equality 



IdT d{A)^ 



0. 



Tda dE 

(The proof of this equality will be given below.) The substitution of Eqs. ( |3-49D and (3-68 
(|347D yields 



3-60) 
3-61) 

3-62) 

3-63) 
3-64) 

3-65) 
3-66) 

3-67) 
3-68) 

3-69) 
to Eq. 



Aa + 0{{Aaf). 



(3-70) 



1 dE /_..o\ , d 
Recalling Eq. ( |3-46D , we finally obtain Eq. (3-38). 

(q-e.d) 

Here, the proof of Eq. ( 3-69| ) is shown. One may find that Eq. (3-69) is equivalent to a Maxwell's 
relation, 
(proof) 

For simplicity, we use the abbreviation A for {A) ^. Then, ^ is a function of {S-B-,a). Since we 
can write 

'dE\ 



A = A{SB,a) 



da J 



Sb 



we calculate 



dA\ ^ /dA_\ /dSB\ 
dEj^ KdSBjAdEj, 
1 d'^E 



TdS-Bda' 
1 dT 
T~da' 



(3-71) 

(3-72) 
(3-73) 
(3-74) 
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(q.e.d) 

3.4. Remark 

In this sections, we have shown that ASb is positive for most probable step processes. One may 
then ask whether or not ASb is positive for arbitrary processes. As one example, one may evaluate 
ASb near quasi-static processes based on several physical assumptions. However, if we consider this 
question from the definition of ASb, it seems hard to obtain any general results. 

Nevertheless, since Sb is equivalent to the thermodynamic entropy, we expect the inequality 

^^B > o{N) (3-75) 



holds for most probable processes in general. We will discuss the validity of Eq. ( 3-75 ) in Section 
6.2 based on Lyapunov analysis of chaotic systems. We now leave statistical mechanics and enter 
Lyapunov analysis. 



§4. Lyapunov analysis 



One of essential features of chaotic systems is the sensitivity of initial conditions. Consider a 
trajectory segment, {r{t), < t < oo}. Almost all trajectories starting from phase space points in 
a neighborhood at r{0) separate exponentially in time from the trajectory r{). Such a behavior can 
be discussed quantitatively by measuring the expansion of vectors in the tangent spaces around the 
trajectory. More generally, we can discuss the time evolution of the A:-dimensional volume element, 
which is given by the exterior product of k independent vectors in the tangent space. (See Appendix 
for basic properties of the volume element and exterior product.) Such an argument includes the 
Liouville's theorem as a special case {k = 2N), which states that the 2N dimensional volume element 
keeps its volume along the trajectory. From the observation for both the cases that k = 1 and 
k = 2N, we expect that the tangent space at each point is decomposed into subspaces associated 
with the expansion ratios. 

Indeed, it has been known that the multiplicative ergodic theorem of Oseledets provides a math- 
ematical description of the naive expectation, Nowadays, the analysis of tangent spaces, which is 
often referred to as Lyapunov analysis, becomes astandard technique to discuss chaos owing to the 
establishment of a numerical calculation method. 

In this section, we pay attention to Hamiltonian systems without parameter change except for 
the final two subsections, and we review the Lyapunov analysis with emphasising its computational 
aspects. 

When the value of a is not changed in time, the evolution map from t = to to t = ti takes a form 
Ut-^-to and satisfies 

r{t) = Ut{r{o)). (4-1) 

The change of the trajectory at time t, Sr{t), against infinitely small change of the initial condition, 
6r{0), is written as 

6r{t) = Utino) + 6r{o)) - c/t(r(o)), (4-2) 

= T{t,r{0))5r{0). (4-3) 

T{t,r{0)) is called the linearized evolution map and is calculated by numerical integration of the 
linearized evolution equation. Note that the matrix T{t, r{0)) is determined by the trajectory segment 
{r{t'),0 <t' < t}. We analyze the matrix r(i, r(0)) below. 

4.1. Gram-Schmidt decomposition 

Let {ci, 1 < i < 2N} be a set of orthogonal unit vectors given randomly in the tangent space 
at r{0). For a while, we will use the abbreviation T for T(t, r{0)). Since almost all vectors expand 
toward the most unstable direction, the direction of the vector Tei may approach to the most unstable 
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direction when t is sufficiently large. We thus define a unit vector in the tangent space at r{t) as 

/■ = P-.- (44) 

/i is expected to indicate the most unstable direction at r{t) when t oo. Similarly, we define the 
most unstable direction in the orthogonal co-space of /i 

r _ 'Tea- (Tea, /i) ^ 
^'-\re,-iTe,J,)\- 

Repeating the similar consideration, we define the i-th unstable direction 



Since {/j, 1 < i < 2A^} is a set of orthonormal unit vectors in the tangent space at r{t), we can find 
an orthogonal matrix J^{t,r{0)) given by 

/, = ^(t,r(0))e,. (4-7) 

Further, from Eq. ( [4-6[ ), Tci is written as 

T{t, ^{0))e^ = ^ L,k{t, r(0))^(t, r(0))efc, (4-8) 
it 



where Lij is the (z, j)-element of an lower triangle matrix C. Eq. (4-8) is the Gram-Schmidt decom- 
position of the matrix T. Since the diagonal element will be particularly important below, we write 
it explicitly as 

j-i 

Lu = \re,-Y.iTe,J,)f,\. (4-9) 
i=i 

4.2. Convergence property 

As mentioned above, fi indicates the i-th unstable direction only when t — > oo. Let eii:{r{t)) be 
the 'true' i-th unstable direction at r{t). In order to have ej,,(7~'(t)) within a certain accuracy, we 
need to confirm 

d{J^{t,^{0))e^,e,i{^{t))) < e, (4-10) 

where e is a small number related to the accuracy we require, and d{e, e') is the absolute value of the 
sine of the angle between two unit vectors e and e'. 



d{e,e') = ^l-{e,e'r. (4-11) 

However, since we do not have e^i{r{t)) yet, we cannot confirm whether or not Eq. ( [4-lCI| ) is satisfied. 
Then, instead of Eq. ( [4-lCl| ), we check the condition 

d{T{t,rmei,J'{t,r{{)))e'i) < e (4-12) 

for two sets of orthogonal unit vectors {cj, 1 < i < 2N} and {e'j, 1 < i < 2N} which are made 
randomly. When Eq. (4-12) is satisfied, we assume that the true i-th unstable direction is determined 

by 

e«(r(t)) ~.F(t,r(0))ei (4-13) 

within an accuracy we require. 

When we numerically obtain e^,i(J~'(0)) at an arbitrary point r{0), we consider a trajectory 
segment {r{t), —tb < t < 0} and check the condition 

d{J^{h, r{-h))ei),nh, n-hM)) < e, (4-14) 
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for sufficiently large ti,. When Eq. (4-14) is satisfied, we assume 

e*i(r(0)) ~^(4,r(-tb))ei (4-15) 

within an accuracy we require. 

We do not know a mathematical condition under which Eq. ( [4- 121) is satisfied. In the argument 
below, we assume that Eq. ( ^-121 ) is satisfied and that a set of vectors {e^,i{^)} is determined for an 
arbitrary point F. Once e^i{r{0)) is determined, e^i{r{t)) is calculated by 

e„(r(t)) = J^{t, r(0))e«(r(0)). (4-16) 

4.3. Lyapunov vectors 

So far, we have stated that e*j(i^(t)) indicates the i-th unstable direction at r{t). More precisely, 
eiti{r{t)) indicates the most unstable direction in the orthogonal co-space of the subspace spanned 
by {e^j{r{t)), 1 < j < i — 1} in the tangent space at r{t). The term orthogonal co-space in this 
statement leads to a non-favorable property of e^i{r{t)): e<t:i{r{t)) does not satisfy 

r(t, r(o))eri(r(o)) oc e„(r(t)) (4-i7) 

except for the case i = 1. This seems a little bit strange, because the unstable nature should be defined 
as something consistent along the trajectory. Then, we define a set of vectors {S,i{r(t)), I < i < 2N} 
which satisfies the two conditions. The first condition is the transitivity 

T{t,^{o))U^m^^^i^{t)), (4-i8) 

and the second condition is that the vector space generated by {e*j, 1 < j < i} is spanned by 
{Cj, 1 < i < The second condition is expressed by 

Y,Aij^,{r{t))=e,i{r{t)), (4-19) 
j 

where Aij = for i < j, and Aij is regarded as the (zj)-element of a lower triangle matrix A. 
Now, we define the i-th expansion factor Ai{t, r{0)) in the i-th unstable direction 

T{t,r{o)Mr{o)) = Ai{t,r{o)%{r{t)). (4-20) 

We call the i-th Lyapunov vector. In order to determine uniquely the value of Ai, we assume 
the normalization condition that the volume of the parallelaid made by {S^j, 1 < j < i} is the unity. 
This condition is expressed by 

|A^=iC,-| = l (4-21) 

for 1 < i < 2N. (See Appendix.) We also assume that An is positive. Under these conditions, we can 
prove that the i-th expansion factor Ai{t,r{0)) is calculated by the Gram-Schmidt decomposition 
Eq. dU) with e,i{r{0)). 
(proof) 

From Eq. ( |4-8D , we have 

T(t,r(0))e„(r(0)) = J2L,k{t,r{0))T{t,r{0))e.k{rm. (4-22) 

k 

Using Eqs. ( |4-19D and ( [4 -201 ), we rewrite the left-hand side of Eq. ( [4-22| ) as 

J2Ajim)Tit,r{0mjir{0)) (4-23) 
j 

= j2Mm)Mt^nomj{m) (4-24) 

j 

= Y.Aij{r{0))Aj{t,r{0)){Air{t))-^)jke,kim) (4-25) 

jk 

= J^Aj{r{o))Aj{t,r{o)){A{r{t)r'),kHt,rme.k{r{o)). (4-26) 

jk 
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Comparing the right-hand side of Eq. ( |4-22|) and Eq. (4-26), we find 

Uk{t, r(o)) = A^,{rmMt^ r(o))(^(r(t))-i),,. (4-27) 

i 

Further, from Eqs. (^191) and ( F2l|) , we can easily see 

\Aii\ = 1 (4-28) 

for 1 < i < 2N . (See Appendix.) Since An is assumed to be positive, An = 1. Then, Eq. ( [4-27D 
yields 

Ai{t,r{^))=Ln{t,r{^)). (4-29) 
In this way, the i-th expansion factor can be calculated numerically. 

(q.e.d) 

4.4. Lyapunov exponent 

The i-th expansion ratio Xi{r{t)) at r{t) is defined as 

^^M^f^ = A.(r(t)M.(t,r(o)). (4-30) 

The long time average of the i-th expansion ratio Xi{r(t)) is called the i-th Lyapunov exponent, 
which is given by 

Xi= lim - r dtXi(r(t)), (4-31) 
T^oo r Jo 

= lim -logAi{T,r{0)). (4-32) 

Xi is sometimes called the 'local' Lyapunov exponent because Aj depends on r{0). However, from the 
ergodic theorem, Aj has a same value for almost all r(0) with respect to the microcanonical measure. 
Since we assume the ergodicity of the microcanonical measure, we do not take care of the local nature 
of the Lyapunov exponent. 

As clearly seen from the method of construction of Lyapunov vectors, we find 

Ai > A2 • • • > X2N- (4-33) 

In Hamiltonian systems, there are at least two zero Lyapunov exponents whose Lyapunov vectors 
indicate the normal direction of the energy surface and the tangential direction of the trajectory. In 
the argument below, we assume that there are Np positive Lyapunov exponents. Unless the system 
has a further conservation law such as momentum conservation, Np equals to — 1. 

The information loss rate at r{t), h{r{t)), is defined as the sum of the expansion ratios with the 
positive Lyapunov exponents. 

him) = E ^^(nt)), (4-34) 

Ai>0 
Np 

= YWt))- (4-35) 

i=l 

Notice that h{r{t)) represents the volume expansion ratio of the Ap-dimensional unstable space. 
That is, h{r{t)) is rewritten as 

h{r{t)) = I log I A^i^ T{t, r{o)Mm)\. (4-36) 

The long time average of the information loss rate, h, has the same value for almost all initial 
conditions with respect to the microcanonical measure. ItJias been known that h is identical to the 
Kolmogorov-Sinai entropy when the system is hyperbolic. ElP 
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4.5. Contraction ratio 

Let us recall that the expansion factor Ai is calculated by the Gram-Schmidt decomposition under 
the normalization condition Eq. ( 4-21 ). However, this normalization lacks the balance between the 
unstable and stable directions. Since Hamiltonian systems possess a time reverse symmetry, such 
unbalance will cause theoretical complicatedness. In order to recover the symmetry, we introduce a 
new set of vectors 1 < « < 2A^} given by 

Cf)=Ci67V-^+l, (4-37) 
where Cj is a positive number determined so as to satisfy 



A;=ief 1 = 1 (4-38) 



for 1 < i < 2N . The set of vectors {s!f\ 1 < "i < 2A^} is made to respect stable directions from the 
most stable one. 

(s) is) 

We now define the contraction factor Ai and contraction ratio as 

T{t,m)^'\m) = ^ &\r{t)i (4-39) 
\^\nt)) = j^\ogA^\t,m). (4.40) 

As will be seen in the next section, the contraction ratio is related to the expansion ratio of the 
time-reversed trajectory and this relation plays a role in simplifying arguments. In particular, the 
following estimation will be utilized. 



i=l i=l 



E Wi-. = - E + Jt^iN). (4-41) 



Here, the last term represents the time derivative of a function whose value is much smaller than 
when N ^ oo. Note that the left-hand side and the first term of the right-hand side are the order of 
N. 

(proof) 

Substituting Eq. (|]3^) intro Eq. (4-39), we have 

Ci{r{0))A2N+i-iit, m) = Ci{r{t)). (4-42) 

The time derivative of the logarithm of the both-hand sides yields 

W-.(rW) = -A!"(m) + liog(|™>). 

We thus obtain 

E w-,(rw) ^ - E A<"(m) + 1 E (ffl) . 



(4-43) 



1=1 i=l i=l 



(4-44) 

Let us evaluate the second term of the right-hand side of Eq. ( 4-441) . We first define an 'angle' 
4>i as 

I A^^i Cj\ = I ^'=i' 6-11 A]^2N+i-i Cj\ sm</>^, (4-45) 

where < </>j < 7r/2. (See Appendix.) By using the normalization condition of {^i}, we rewrite Eq. 
(^45|) as 

\/\]=2N+i~i^j\ sin = (4-46) 
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Using Eqs. ( 4-37 ) and ( 4-3S ), we obtain 

ci- ■■ Ci = sin 4>i. (4-47) 

This leads to 

where we have assumed 

sin(0jv,(r))>O(exp(-Ar)), (4-49) 

which may be ensured by the condition that the unstable and stable manifolds intersect transversally. 

(q.e.d.) 

4.6. Weight on trajectory segments 

We consider a weight on the trajectory segment {r{t), < t < r}. The weight, W{{r(t), < 
t < r}), is a conditional probability finding trajectory segments remaining in a small tube around 
{r{t), < t < r} when the initial condition is chosen in a small region around r{0). More explicitly, 
the weight W is defined in the following way. 

Suppose that the phase space is decomposed into small cells {Aj} with a sufficiently small size e 
and that r(0) is included in the i-th cell Then, the number of cells which intersect with Ur{^i), 
N(t, e), can be counted. We can choose the value of e so that the region Uri'^i) remains in a linear 
regime around ^7^(1^(0)). This condition may be given by 

e«ec(r), (4-50) 

where the value of ec(r) is determined by nonlinear properties of dynamical systems. Under this 
condition, N(t, e) measures the number of distinguishable trajectory segments starting from the 
neighborhood of r{0). Therefore, we define the weight as 



Warit), 0<t<r}) = ^^^. (4-51) 



Then, we can show 



warn, o<t<T}) = I A£r(-r,r(T))c.(r(T))| (4.52) 

in an appropriate limit of large r and small e. (Since edr) for the limit r oo, we need to take 
care of a delicate problem of double limits. Nevertheless, we assume simply that we can choose an 
appropriate asymptotic limit.) 
(proof) 

Consider the time evolution of a small region Z\j. The region expands and contracts in the 
unstable and stable directions, respectively. After a sufficiently long time, the region almost collapses 
into the A^p-dimensional unstable manifold, and intersects with cells in the unstable directions. Since 
the A'p-dimensional volume element in the unstable manifold at F^t) is written as A^]^^j(-r(r)), we 
expect 

Nir,e)^ , |A£e.(r(.))| (,.,3^ 

= |A£T(-r,r(r))C,,(r(r))ri (4-54) 

in an appropriate limit of large r and small e. Substituting this into Eq. ( 4-51| ) leads to Eq. ( [4-52| ). 

(q.e.d) 

Further, by the relation Eqs. ( [4-36| ), Eq. ( [4-52| ) becomes a simpler form 



W{{r{t), < t < r}) = exp(- r dth{r{t))). (4-55) 

JO 
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4.7. Time dependent case 

To this point in this section, we have assumed that the value of a is not changed in time. In 
this subsection, we briefly discuss Lyapunov analysis for systems with a time dependent parameter. 
When the value of a depends on t, the evolution map from t = to to t = ti depends on the absolute 
time to and ti. Therefore, it takes the form Ut-^^toi ™d the linearized evolution map is written as 
T{ti,to;r{to)). 

The Lyapunov analysis in such a case may be reconsidered carefully. However, we do not need 
general arguments. In the systems in question, the value of a is changed during a finite time interval 
[Ti, Tf], where <^ n < tj <^ t. Therefore, for example, the i-th Lyapunov vectors at r{0) and -r(r) 
can be defined as ^i{r{0)) and ^i(/^(r)), respectively. 

Although the expansion factors, Lyapunov exponents and information loss rate do not make a 
sense in general, the argument on the weight W is still valid. We can write 



W{{r{t), 0<t<r}) = I A^^ mr;r{T)Mr{r))\ 



(4-56) 



for sufficiently large r and small e. We also define the actual information loss rate as the generalization 
ofEq. ( F36| ) 

d 



Kit,r{o)) 



log\Al-,'T{t,0;rmUm)\- 



dt 



(4-57) 



4.8. Liouville's theorem 



In this subsection, we review a proof of the Liouville's theorem which states that the 2N- 
dimensional volume element keeps its volume along the trajectory. It is important to understand 
that the Liouville's theorem holds even when the value of a is changed in time. 

(proof) 

We have the Hamiltonian equation 



dFjt) 
dt 



-J 



dH{r,a{t)) 



dr 



r=r{t) 



(See Eq. ( |2-8D in Section 2.) Since the linearized evolution equation is written as 



d6r{t) 
dt 



-J 



d'^H{r,a{t)) 



dFdr 



srit), 



r=r{t) 

the linearized evolution map T{t,0; r(0)) satisfies the equation 

where ;B is a symmetric matrix. We then obtain 

dt dt dt 

= -{T^BJ^JT + T^JJBT) 

= 0, 



where we have used the equality 

J^J = -J J = 
Since T(0,0;r(0)) = 1, Eq. (4-63) leads to 

T^JT = J. 
The determinant of the both-hand sides gives 

det [T^T] = 1 



1. 



(4-58) 
(4-59) 

(4-60) 

(4-61) 

(4-62) 
(4-63) 

(4-64) 
(4-65) 

(4-66) 
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Let {cj, 1 < i < 2A^} be an orthogonal set of unit vectors defined in the tangent space at -r(O). 
The time evolution of the 2N dimensional volume element A^^^ej is given by Af^^Tci, and its volume 
is calculated as 

I A^^i Te,| = VdetTTt (4-67) 
= 1. (4-68) 

(See Appendix.) Therefore, the 2A^-dimensional volume element keeps its volume along the trajectory, 
(q.e.d.) 

Further, using the Liouville's theorem, we can prove that the equality 

2N 

J2X,{r{t)) = (4-69) 

i=l 

holds when the value of a is not changed in time, 
(proof) 

Since the the value of a is not changed in time, we obtain 

2N r 2N 

Y,Ur{t)) = -\ogY^Mt,m) (4-70) 

i=l i=l 

= |log|A2^iT(t,r(0))e.(r(0))| (4-71) 
= (4-72) 
where the Liouville's theorem is used to obtain the last line. 



(q.e.d) 



§5. Reversibility 



5.1. Reversibility in time evolution 
We define a matrix TZ as 

{nr)i = qi, (5-1) 
{nr\+N = -Pi, (5-2) 

where 1 < i < A^. The matrix IZ corresponds to the time reverse operator acting on the phase space 
point. We assume that the Hamiltonian under consideration possesses the time reverse symmetry 

H{nr,a) = H{r,a). (5-3) 

Let Utfi and Utfl be the evolution maps for Hamiltonian equations with a() and q(), respectively, 
where we have defined the time-reversed protocol of the parameter change a{) as 

a{t)=a{-t). (5-4) 

Then, owing to the symmetry property Eq. the identity 

Ut,o = nii-tfln. (5-5) 

holds. 

(proof) 

Let {r{t)} and {r{t)} be trajectories given by 

r{t) = Utflirm, (5-6) 
r{t) = Utflirm, (5-7) 
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where r{0) and r{0) are the initial conditions which satisfy the relation 

f (0) = 7^^(0). 

From Eq. (5-7), we obtain 



dr{-t) dH{r,a{-t)) 



dt 



-J 



dr 

dH{r,a{t)) 



r=f{-t) 



dr 



r=r{-t) 



(5-8) 

(5-9) 
(5-10) 



where we have used the equation of motion in the form Eq. ( |2-8| ) with the matrix satisfying Eq. 
(|2-9|). On the other hand, Eq. (5-6) leads to 



7^ 



dr{t) 

dt 



-nj 
jn 
J 
J 



dH{r,a{t)) 



dr 

dH{r,a{t)) 



dr 

dH{r,a{t)) 



d[nr) 

dH{nr,a{t)) 



J 



d{nr) 

dH{r,a{t)) 



r=r{t) 

r=r(t) ' 
r=r{t) ' 



or 



r=nr{t) 



Here, the second line is obtained by the relation 

nj + jn = 0, 



(5-11) 
(5-12) 
(5-13) 
(5-14) 
(5-15) 

(5-16) 



the third line is derived from the relation TZTZ = 1, and the equality of the forth line comes from the 
symmetry property Eq. (|5-3D . 

Comparing Eqs. (5-10) and (5-15), we find that r{—t) and lZr{t) obey the same evolution 
equation. Recalling the relation for the initial conditions Eq. ( |5-^ ), we conclude 

(5-17) 

(5-18) 
(q.e.d) 



r{-t) = nr{t). 

By using Eqs. (5-6) and (5-7), Eq. ( ^-171 ) is rewritten as 

c/_t,o(^r(o)) =7^c/t,o(^(o)). 

Since r{Q) is arbitrary, Eq. (5-5) holds. 

In the argument below, r[t) will be assumed to be given by Eq. ( |5-17 ). 

5.2. Reversibility in Lyapunov analysis 
First, from Eq. ( ^-SD , we have 

T{t, 0; r(o)) = nf{-t, 0; ^(o))7^, 



(5-19) 



where T{t,0; r{0)) and T(t, 0; 7^(0)) are the linearized evolution maps around the trajectories r{) 
and -TO. In particular, when the value of a is not changed, the equality T = T holds. We then prove 
the identities 



Ciir) = nc^\nr), 
x,{r) = x'f\nr), 

h{r{t)) - h{n-t)) = ^^o{N). 



(5-20) 
(5-21) 

(5-22) 
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(proof) 

Using the matrices defined as 



we can write T(t, -r(O)) in the two forms 



Xi,{r) = (5-23) 

x^;\r) = (5-24) 



T{t,r{o)) = x{r{t))M{t,r{o))x{r{o))-\ (5-25) 
T(t,r(o)) = x^'\r{t))M^'\t,r{o))x^'\r{o))-\ (5-26) 

where >l(t,r(0)) and M^'\t, r{0)) are diagonal matrices whose {i, i)-elements are given by Ai{t, r{0)) 
and ylf^(t,r(0))~\ respectively. Using Eq. (5-26), we rewrite the right-hand side of Eq. ( |5-19D as 

7^;t'(^)(f(-^))>lW(-^,^(o))A'(^)(f(o))-l7^, (5-27) 
= nx'-'\nr{t))M'-'\-t,r{o))x^'\nr{o))-^n, (5-28) 

where we have used T = T. Comparing Eq. (5-28) with the right-hand side of Eq. (5-25), we obtain 

x{r) = nx^'\nr), (5-29) 
M{t,r{o)) = M^-'\-t,nr{o)), (5-30) 

where notice the normalization conditions of and £,1'^^ given by Eqs. ( 4-21D and ( [4-38| ). 

Equation (5-29) is equivalent to Eq. (5-20), and Eq. (5-30) leads to Eq. (5-21), because of the 
equality 

±iogA'f\-t,nm)-' = x^\n-t)), (5-31) 

= x^\nr{t)). (5-32) 

Furthermore, h{r[t)) is expressed in terms of {a|*^} in the following way. 

h{r{t)) = J2x.{r{t)), (5-33) 
1=1 

Np 

= -E^2iV+*-i(r(t)), (5-34) 
1=1 

= Y.xl^\r{t)) + -o{N), (5-35) 
1=1 

where the second and third lines come from Eqs. ( ^-691 ) and Eq. ( 4-41 ), respectively. On the other 
hand, from the symmetry property Eq. (5-21), h{r(—t)) is written as 



Mr(-t)) = ^A,(f(-t)), (5-36) 

i=l 
Np 

= g\(7^^(^)), (5-37) 

i=l 

Np 

= J2^^\r{t)). (5-38) 
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Comparing Eqs. (5-35) and (5-38), we obtain Eq.(5-22). 

(q.e.d.) 

Using these identities, we can express the weight on the trajectory segment by the actual infor- 
mation loss of the time reversed trajectory 

W{{rit), < t < r}) = r dtha{-t, f(-r)) exp{o{N)), (5-39) 

Jo 

where the actual information loss rate along the time-reversed trajectory ha is defined as 

ha{t, r(-r)) = I log I A^l^ f{t, -r; r(-r))e.(r(-r))|. (5-40) 

(prove) 

Let us recall the expression of the weig ht Eq. (|]5|). 

want), < t < r}) = I a£ T(0,r;r(r))e.(r(r))|. (5-41) 

The right-hand side is rewritten in the following way 

I A£7^t(0,-T;f(-r))7^7^ef^(7^^(T))| (5-42) 

= I A£r(0,-r;f (-T))4(^)(f (-r))| (5-43) 

= I A^I^i t(0, -r; r(-r))6iv+i-^(r(-T))| exp(o(7V)) (5-44) 

= I A^^^f(0,-r;fi-r)Mfi-r))\expio{N)), (5-45) 



r dthai-t,r{-T))expioiN)) (5-46) 
Jo 



where the third line is obtained by using an argument in section 4.5, and the forth line comes from 
the Liouville's theorem. 

(q.e.d) 

Similarly, the weight on the time reversed trajectory segment W{{r{t), — r < t < 0}) is written 

as 

W{{f{t), -T < t < 0}) = r dthait, r(0)) exp(o(7V)). (5-47) 

Jo 

(See Eq. ( ^ ) for the definition of ha{t,r{0)).) 

5.3. Reversibility paradox 

Suppose that there is a trajectory segment {r{t), < t < r} from an energy surface Uq to Ui. 
Then, the time reversed one {r{t), — r < t < 0} goes from Ei to Eq. One may wonder how this 
fact is compatible with thermodynamic irreversibility. The essentially same question was proposed by 
Roschmidt, and has been known as the reversibility paradox. A standard answer may be replying 
on the operational impossibility of the time reverse operation F — > TZF. If we were allowed to operate 
the system by using the result of the observation of the trajectory, we could perform the time reverse 
operation. This consideration is related to the Maxwell's demon's problem. Ed-* However, the time 
dependence of a is given without any references of trajectories. Thus, in our problem, the time 
reverse operation cannot be realized by a() and the Maxwell's demon problem does not appear. 

However, still the paradox is not resolved completely. In order to be compatible with thermody- 
namic irreversibility, there should be asymmetry between the trajectory segment {r{t), < t < r} 
and the time-reversed one {r{t), — r < t < 0}. The asymmetry cannot come from a purely me- 
chanical consideration. We must consider the measure for the ensemble of the initial conditions 
of the time-reversed trajectory segment {r{t), —t < t < 0}. This ensemble, TV, is defined as a 
{2N — l)-dimensional set which satisfies 

Uo,-r{^r) = So- (5-48) 
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From the reversibility relation Eq. ( |5-5| ), we obtain 

Tr = nUr,o{^o). (549) 

Owing to the chaotic nature, T^- becomes a quite complicated set as r is large. Here, we describe the 
set T-r informally. We focus on the thermodynamic limit so that the structure of TV is clearly seen. 
Suppose that the most probable processes Eq^^Ui and IJi^^Hq are realized by the protocols of 



the parameter change q() and a{), respectively. Then, from Eq. (5-48), Tt-DUi becomes dominant in 
X!i with respect to the microcanonical measure for X!i. On the other hand, from Eq. ( ^-491 ), TV H Ui 
becomes dominant in T^- with respect to the microcanonical measure for Eq. One may wonder that 
these two statements are apparently contradictory. However, we should note that the measures are 
different when we observe the set T^. Here, it is worthwhile noting that n Ui is not dominant in 
El with respect to the microcanonical measure for Ei, when Ei ^ Ei. Therefore, we can imagine 
that the set has a fine structure in energy surfaces apart from Ei. In order to represent this 
heterogeneity quantitatively, we define a measure p. for the set Tt as 

-fAfPf ^^T^ r MMr{-r))mZUr,oiEoo6E)) 

fi{Ae{r{-T));Tr) = hm ^-prr ( r. xt^w ' ^^'^^ 

SE^o fii^[KUrfl[Eoo dh)) 

where /iL denotes the 2A^-dimensional Lebesgue measure, A^{r{—T)) is a small region with a size e 
including r), and Eq o 5E represents a set of energy surfaces with width 5E. (See section 2.4.) 
We then expect that Ji{A^{r {—t));Tj-) for r(—T) G Ei is much larger than /i(Z\e(/'(— r); T,-) for 
r(-r) G El. 

5.4. Reversibility in probability 

In spite of the asymmetry between the two sets Eq and Tr, from the reversibility of the time 
evolution, there is a one-to-one correspondence between a set of trajectory segments from Eq to TZTt 
and a set of time-reversed trajectory segments from TV to Eq. 

We now derive the reversibility relation coming from this correspondence. In Section 4.7, we 
discussed the weight on trajectory segments. The weight W{{r{t), < t < r}) is a conditional 
probability finding trajectory segments remaining in a small tube around {r{t), < t < t} when 
the initial condition is chosen in a small region A^{r{0)) around r(0). Then, 

fi^ciMrm-, EQ)W{{r{t), < t < r}) (5-51) 

is a probability finding a trajectory segment in a small tube around {r{t), < t < r} of all trajectory 
segments from Eq to TZTr- The probability, from the one-to-one correspondence mentioned above, 
should be equal to a probability finding a trajectory segment in a small tube around {r{t), — r < 
t < 0} of all trajectory segments from TV to Eq. The latter probability is written as 

fi{A,{f{-T));Tr)W{{f{t), -T<t<0}). (5-52) 

Therefore, we obtain the relation 

^^e(Z^e(r(0)); EQ)W{{r{t), 0<t<T}) = fiiAAr(-T)); Tr)W{{r{t), -T<t< 0}). (5-53) 

This relation will lead to an important equality related to thermodynamic irreversibility. 

§6. Irreversible information loss 

6.1. Definition 

Let us define the irreversible information loss I as 

T( nn^^ . ^ warjt), -T<t<o}) 

= want), < t < r}) ' ^'-'^ 
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where the right-hand side depends on (r, /^(O)) because the trajectory is given by a solution to the 
deterministic evolution equation. Using Eqs. ( |5-39| ) and ( ^-471 ), we can write / as 



/(r, r(0)) = / dt[ha{t, r(0)) - ha{-t, n-T))] + o{N). (6-2) 
Jo 

The expression may represent the meaning of the term 'irreversible information loss'. Then, from the 
relation Eq. ( 5-53 ), we obtain 

/ f,{A,{r{-T));Tr)= f /i^c(/^.(r(0)))exp(-/). (6-3) 

J TV ^0 

By the normalization condition of the probabihty, we have the equahty 

(exp(-/))o = 1, (6-4) 
where ( )q denotes the average by fi^cidF; Uq). Using the Jensen's inequality 

(exp(-/))o<exp(-(/)o), (6-5) 

we obtain 

> 0. (6-6) 

This inequality suggests that the irreversible information loss / has a certain relation with thermody- 
namic irreversibility. One may find that the argument has some similarity with that by Jarzinski.H) 
In order to discuss the convergence of /(r, r{0)) for r — > oo, we evaluate the value of 

hm 1^ = hm [ha{T,r{0)) - ha{-T,f{-T))]. (6-7) 

Since the value of a is changed in a finite time interval, A,^]^T(t, 0; i~'(0))^i(/^(0)) approaches to the 
unstable manifold around r{t) when t ^ tj. We thus obtain 

lim [ha{T, r(0)) - ha{-T, f{-T))] = lim [/i(r(r)) - hir(-T))] (6-8) 

= j^o{N), (6-9) 

where we have used Eq. (5-22). We further assume that this convergence is so fast that the time 
integration becomes a finite value. Then, there exists a function / such that 

hni hm ll(r, r(0)) = 7(r(0)). (6-10) 

T^OO N^oo iV 

6.2. Most probable value 

We define the most probable value of l{r{0)) based on the assumption of the large deviation 
property: 

Let Uj^ijj] SQ,a{))dijj be a probability such that I{T,r{0))/N takes a value in [ip,tp + dip] when 
the initial equilibrium state Uq and the protocol of the parameter change q;() are given. Then, IIi is 
written in the form 

77/(^; Uo, aO) ~ exp(-A^07(^; Uo,a{))), (6-11) 

in the thermodynamic limit. 

The probability density 77/ is induced from the microcanonical measure for the initial conditions 
on Ef). The rate function (f)j is a convex and non-negative function, and the most probable value 
satisfies 

cl)i{h;Lo,a{)) =0. (6-12) 
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Here, Eq. (|6-6| ) leads to the inequality 

/* > (6-13) 

for an arbitrary most probable process. 

We then try to find a relation between and ASb- First, let us note that (/)q is rewritten as 



So /^mc((^e(-r(0));Z'o 



(6-f4) 



where we have used Eq. ( |5-53| ). We consider the thermodynamic limit in the expression Eq. ( |5-14 ), 
although the argument is not completely formalized yet. 

When the most probable process i7o— is realized, TV n Si is dominant in TV with respect to 
the microcanonical measure on Uq. Also, from the mixing property, Tt D Ui may be identified with 
Ui in a coarse graining description of the phase space. Thus, we can expect that fl{A^(r(—T));TT-) 
in Eq. ( |6-14 ) may be replaced by ^mc(^e(-^(— t)); i^i) exp(o(A^)), in an appropriate limit of small e, 
large r, and large A^. When this expectation is valid, we can rewrite Eq. ( |6-14 ) as 



So 



(Z\,(r(0));i7o)log 



/^mc 



+ oiN), 



Thus, we obtain 



^^c(A(r(0)); Uo) log ^ + o{N). 
So 1^1 1 



hm — log— — 

TV^oo A 1 2^0 1 



(6-15) 
(6-16) 

(6-17) 



This equality Eq. ( |6T7|) shows that the value of 7* is determined by initial and final states irrespective 
of details of the process Uq^^Si. Furthermore, from Eqs. ( 3-28| ) in Section 3.2, we can express /* 
in the form 



f r 1 1 ^1 
= lim — log——, 

= hm hsBiUi) - Sb{Eo)]. 

TV— >oo iV 



(6-18) 
(6-19) 



That is, the most probable value of the irreversible information loss is equal to the change of the 
Boltzmann entropy per a unit degree. Also, the inequality Eq. ( |6T3| ) can be read as 

Sb{IJi) > Sb{IJo) + o{N) (6-20) 

for an arbitrary most probable process Uq^^^Ui. This implies that the Boltzmann entropy satisfies 
the entropy principle in thermodynamics. 

6.3. Fluctuation theorem 

Let us consider a probability ni{^;TT,aO)d^ such that the irreversible information loss / takes 
a value within [Nip, N{ip + cZV')] when the measure of the initial conditions is assumed to be fi and 
the time-reversed protocol a{) is given. 77/(V'; TV, is written as 



77(V;r„a())dV' 



/i(z\,(f(-T)); rOE(v^ < /(r, r{-T))/N < + dV), 



(6-21) 



where E(*) takes the value 1 when the statement * is true. The right-hand side of this expression is 
rewritten in the following way. 



exp(AfV) / ^mc(/^e(r(0)))E(-V' -dip< I{t, r{0))/N < -i,) 
J So 

eKp{N7p)ni{-7p; Uo, a{))dip, 



(6-22) 

(6-23) 
(6-24) 
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where we have used Eq. ( |5-53| ) in order to obtain the second hne. Therefore, we obtain 



exp(iV^). (6-25) 



This may be called the fluctuation theorem in Hamiltonian systems with a time-dependent parameter. 

We could not derive a useful expression of Eq. (|6-25| ) in the thermodynamic limit. We explain the 
reason. Suppose that the most probable process Si^^Sq is realized by the time-reversed protocol 
a{). Then, the dominant region of with respect to the measure jl, which contributes Uf^ip; TV, dO) 
much, is around the energy surface IJi. However, if 7^ was replaced hy IJi, ip in Eq. ( |6-25| ) could not 
be substituted by, for example, the most probable value /=„ for Uq^^Ui. This loses the significance 
ofEq. ( |6^ ). 

When we are concerned with an infinitely small step process, we can derive the fluctuation- 
response relation from Eq. (6-25). In such a process, T^- may be replaced by Uq at the lowest order 
approximation. Then, since 77/ may be approximated by a Gaussian distribution for large N, we can 
write 

log nii-i;; Eo,aO) = -Arir_L_^ + o{N), (6-26) 
Iog77,(^;i7o,«()) = -A^ ^^^^2 +o{N). (6-27) 



The fluctuation theorem Eq. ( 6-25 ) leads to 

- N ^'^'i*^' + iV ^^+f^' = iVV. + o{N). (6-28) 

2C7^ 2(7'^ 

Since /* and 7^ are infinitely small, Eq. ( |6-28 ) may be valid for arbitrary in a finite range including 
1^ = 0. Thus, we have 

= (6-29) 
f7'2 = a\ (6-30) 

Substituting these equalities into Eq. ( 6-2^ ), we obtain 

Nh = y(T^ (6-31) 

= ^ (6-32) 
= i((7-iV7.)2). (6-33) 

Comparing Eq. (6-33) with Eq. ( |3-44| ) in Section 3.4, we find that this result is consistent with Eq. 
(6-19). 

§7. Excess information loss 

At each point r{t) in a trajectory segment, {7^(t),0 < t < r}, we can consider a Hamiltonian 
system defined on the energy surface ^{t) by virtually fixing the parameter value to a{t). Then, as 
discussed in Section 4.4, we can calculate the information loss rate, h{r; Ij{t)) at F £ ^(t) in this 
virtual Hamiltonian system. We define the excess information loss rate as □ 

/iex(t, 7^(0)) = h,{t, r(0)) - h{r{t); U{t)). (7-1) 

* We obtained the idea of the excess information loss from a paper by Oono and Paniconi,B where they defined 
the excess heat in constructing steady state thermodynamics. 
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Further, the excess information loss H^^ is defined as the time integration of /lex 

//ex(r,r(0)) = r dtK,{t,r{0)). (7-2) 
Jo 

Similarly, the excess information loss rate at r{—t) in the time-reversed trajectory is given by 

/iex(-t, n-T)) = K{-t, r{-T)) - h{f{-t); U{t)), (7-3) 
where r{—t) G ^{t), and the excess information loss in the time-reversed trajectory is written as 

^ex(r,r(-r)) = r dth,^{-t,f{-T)). (7-4) 
Jo 

Using these quantities, we rewrite the irreversible information loss / as 

/(r, r(o)) = dt[h^{t, r(o)) - K{-t, r(-T))] + o{n), (7-5) 

dt[heAt, r{0)) - he^-t, r(-r))] + o{N), (7-6) 
= H,,{t, r(0)) - i7ex(r, r(-T)) + o(iV), (7-7) 
where we have used the equality 

him- m) = Hn-t); m) + j^o{n). (7-8) 

(See Eq. (5-22).) Further, through the definition of Hex:Tev 

i?ex:rev(T,r(0)) = ^[H^^ir, T {0)) + H.^ir, f {-t))], (7-9) 

Eq. (7-7) becomes 

h{T, r(0)) = Fex(r, r(0)) - i/ex:rev(r, r(0)) + o{N). (7-10) 

In the viewpoint of numerical calculation, the excess information loss is a more tractable quantity 
than the irreversible information loss, because H^xiT, r{0)) converges to a certain value i/ex(oo, r{0)) 
when T oo. This convergence can be expected from a fact that he^it, r(0)) converges to when 
t — Tj — > oo. Similarly, we expect that Hex{T,r{—T)) converges to a certain value Hex{oo, r{—oo)) 
when Tj oo. (Note that r ^ oo when Tj oo.) Also, i?ex:rev(oo, -^(0)) is determined. 

Let us consider the average over the initial conditions sampled from the microcanonical ensemble 
on the energy surface Sq. From Eq. ( [7T0D , we have 

^Nh = (i/cx>o - (^cx:rcv>o + o{N) (7-11) 

in the thermodynamic limit, where (-ffex)o is the average of Hcx{oo, r{0)). Since ifcx(oo, -^(0)) can be 
obtained numerically without referring the time-reversed trajectory, (i?ex)o is a directly measurable 
quantity. Although (-f^cx:rev)o is not easily obtained numerically, this may be expected to have a 
certain relation to the quasi-static excess information loss Hex-.qs, which is defined as 

da 



/•oo fin 

^ex:qs = / dt — <PiU{t)), (7-12) 

Jo at 



where the quantity <P[X!{t))da is the excess information loss calculated under the assumption that 
the equilibrium state is virtually realized at each time t along the trajectory r{). (Recall a similar 
discussion below Eq. (|3-31| ).) Notice that -ffexiqs becomes the real excess information loss when the 
process is quasi-static. 
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We are going to discuss the relation between i/exirev and i?ex:qs- We consider a step process 
realized by an infinitely small parameter change a — > a + Aa at t = Ti. Then, the quantity ^{^q) is 
given by 

(//ex)o = <^(^o)^« + 0{{Aaf). (7-13) 



Also, using Eqs. (p-4Cl|) and (6-19), we obtain 

h = 0{{Aaf). (7-14) 

Therefore, we find 

(^cx;rev>o = ^{^0)^0^ + 0{{Aaf). (7-15) 

Since a quasi-static process can be realized by repeating an infinite number of infinitely small step 
processes, (-f^ex;rev)o ^ quasi-static process ^ Ei is written as 

(^^cx;rcv)o= Hda^iS^^)), (7-16) 

where S(^a) is the equilibrium state such that 

^0 = ^(qo) ^(a)- (7-17) 

Then, Eq. ( [7T6| ) implies 



Note however that the validity of Eq. ( 7-18 ) is ensured only for quasi-static processes. Nevertheless, 
we assume that Eq. ( |7T8[) h olds at least near quasi-static processes. Based on the assumption, the 
right-hand side of Eq. ([7-l"l| ) is calculated numerically without referring time-reversed trajectories, 
and from Eqs. (6-19), (|7T1| ) and (|7T8|) , we obtain the expression 

i {ASb)o = (^ex)o - (^cx:qs>o + o{N). (7-19) 

7.1. Minimum principle 

Applying the inequality Eq. (|6-13| ) to the expression Eq. (|7-11| ), we obtain 

+ o{N). (7-20) 

This inequality implies that the excess information loss must not be lower than its reversible part. Such 
a phrase reminds us the minimum work principle in thermodynamics with an isothermal environment, 
which states that the work done by external agents must not be lower than the quasi-static work. As 
discussed in the previous subsection, /7ex:rev may be related to the quasi-static excess information loss. 
Therefore, the analogy with the minimum work principle may be expected more and Eq. ( 7-20| ) may 
be regarded as the minimum excess information loss principle. However, we do not yet understand 



the significance of the inequality Eq. ( [7- 20]) . We expect that the analysis of subsystems may provide 
us a further insight for Eq. ( 7-20 ). This will be a future problem. 

7.2. Expression of <P 

In this subsection, we derive an expression of ^ in terms of Lyapunov vectors. Suppose that the 
value of a is changed instantaneously from ao to ao + Aa at time t = 0. The trajectory FQ is not 
differentiable at t = 0. We consider the excess information loss for FQ 

POO 

Fex(oo,r(0)) = / dt[h^{f,Fm-h{F{t);Ui)], (7-21) 

JO 



where Ei is an energy surface after the parameter change. 
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{if^^ 1 < ^ < 2N} be a set of Lyapunov vectors at -r(O) in the energy surface Eq. We then 



define a set of vectors {ai{t), 1 < i < 2A^} in the tangent space at r{t) as 



a,(t) = r(t,o;r(o))e, 



(0) 



(7-22) 



where T(t, 0; -r(O)) is the hnearized evolution map along the trajectory in the energy surface Si. Note 
that Oj is not the Lyapunov vector at r{t), because ^f''^ is not the Lyapunov vector in the energy 
surface Ei. 

The i-th Lyapunov vector along the trajectory {/^(t), < t < oo} in the energy surface Ui is 
denoted by ^fV(O)- The expansion factor Ai{t,r{Q)) satisfies 



Using ai and Ai, we can write Eq. ( [7-21D as 



i7ex(oo,r(0)) = hm [log] A£ai(t)| -X]log/li(t,r(0))]. 



(7-23) 



(7-24) 



Let us evaluate the right-hand side of Eq. ( [7-24 ) . We expand '^^ by the set of Lyapunov vectors 



{Cf ^(-r(O)), 1 < i < 2N} in such a way that 



2N 



(7-25) 



Here, the matrix Q is defined at 1^(0) and depends on Aa. Then, from Eqs. ( [7-22 ) and ( 7-25 ), ai{t) 
is expanded in the form 

2N 

t(^)irt,\\ (7.26) 



a^{t) = Y,Qi,A,{t,r{mf\r{t)). 





r- 


=1 








we write 


|A£a,(t)| 


as 






E 

(iir--,JiVp) 


n '3fcife 
fc=i 




k=l 




(7-27) 



where the index jk varies from 1 to 2N . When t is sufficiently large, the contribution from the 
unstable directions becomes dominant. Thus, for sufficiently large t, we derive 



log I ai{t)\ ~ log 



E ■■■ E sgnOi,---,jivJ 



ii=i iiVp=i 



Np 

n Qkjk 

k=l 



l[Mt,m) 



k=l 



Np 



~logdet+Q + ^logyli(t,r(0))], 



i=l 



where 



det+Q = E ■ ■ ■ E sgn(ji, • • • , jA^JQiji • • • QNpj^p 



(7-28) 
(7-29) 
(7-30) 

(7-31) 
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and we have used the normaUzation condition | A^^^ ^i^\r{t))\ = 1. Finally, substituting Eq. (7-30) 
to Eq. ( 1^21) , we obtain 

Fex(oo,r(0)) = logdet+Q. (7-32) 



Therefore, from the definition of <?(Z'o), we have 



(logdet+Q)o 



^iUo)= lim ^ ^ ^ (7-33) 

This expression shows that ^ takes a non-zero value when the unstable manifold varies linearly for 
the infinitely small parameter change. 



§8. Numerical experiments 



Until now, we have developed the theoretical arguments. However, one may point out that these 
lack the mathematical rigorousness. We then present evidences by numerical experiments so as to 
confirm the validity of the theoretical arguments. 

As a direct experimental test of our theory, we should check the relation between the irreversible 
information loss and the Boltzmann entropy. However, unfortunately, we do not yet complete this 
test, because it is hard to measure numerically the irreversible information loss. The reasons of this 
hardness follows. 

First, the time-reversed trajectory is needed in calculation of /(r, /^(O)). This fact causes a 
delicate problem: Suppose that we obtain numerically a trajectory segment {T{t\ < t < r}. Since 
the system is chaotic, this trajectory is not an approximation of the true trajectory starting from 
the initial condition -r(O). However, when a pseudo-orbit tracing property is valid in the system, 
there is a true trajectory which is closed to the trajectory segment obtained numerically. Then, when 
we integrate the time-reversed equations of motion with the initial condition r(—T), the trajectory 
obtained numerically deviates from {r{t), — r < t < 0} due to the orbital instability. Therefore, 
in order to obtain the time-reversed trajectory {r{t), — r < t < 0}, we must store the data of the 
original trajectory. 

Second, even if we obtain numerically /(r, /^(O)), it does not converge to a value for r — > oo, 
because only the extensive part of /(r, /^(O)) converges. Therefore, it is not easy to choose the value 
of r in numerical calculation. In principle, we have only to choose large so that the arbitrariness 
becomes less. However, we need much more time to study the systems with larger N. 

In this paper, instead of the irreversible information loss, we discuss numerically the excess infor- 
mation loss, with particularly focusing on Eq. ( [7-191) . Since Hex{T, r{0)) converges to Hcx{cc, r{0)) 
when T ^ oo, the numerical calculation of the excess information loss may be simpler than that of 
the irreversible information loss. Also, the time-reversed trajectory is not needed in the calculation 
of Hex- 

8.1. Model 

A system consisting of many molecules with short-range repulsive interaction may be the most 
realistic model to study thermodynamic irreversibility. However, since we are concerned with a 
universal aspect of irreversibility, the choice of the system does not matter. Simpler models may be 
better to us. That is the reason why we study numerically the Fermi-Pasta-Ulam (FPU) model.® 
The Hamiltonian of the FPU model is given by 

N , , 

H{{qi}, {Pi};g) = ^[2^' + 2(^^+1 - + f^'^^+i - '^^^'l' (^-^^ 

i=l 

where the value of g is changed in time. That is, a{) in previous sections is identified to g{) in this 
section. 
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The evolution equations for ({gi}, {pi}) are written as 



dt 
dpi 
dt 



Pi 



(gfj+i - Qi) + g{qi+i - qif - {qi - qi-i) - g{qi 



(8-2) 
(8-3) 



We assume periodic boundary conditions, qo = q^ and qN+i = qi in Eq. (8-3). Under the boundary 
conditions, XliPi is a conserved quantity. We assume that the value of ^iPi is zero, for simplicity. 
Then, also becomes a conserved quantity. We assume that the value of Qi is zero. In the 

remaining part of this section, the energy surface with the condition ^ j pi = = is simply 

called the energy surface. We solve numerically Eqs. (8-2) and (8-3) by the 4-th order symplectic 
integrator method E^) with a time step 5t = 0.005. Since we are concerned with the thermodynamic 
limit, we check the N dependence of our conclusions. 

8.2. Lyapunov analysis 

In this subsection, we assume that g takes a constant value, say go. Let Eq be the energy. When 
-EqS'o is sufficiently large, the system with large exhibits high-dimensional chaos. As an example 
of such parameter value set, (E'cffo) = (1-0, 10.0) is assumed. 

We first check the mixing property with respect to the micro-canonical measure by discussing a 
relaxation behavior. (See the last paragraph of Section 2.3.) Figure 1 shows an example how the 
average of A relaxes to the equilibrium value when the initial conditions are sampled from an artificial 
ensemble we assumed. As far as we checked, we observed a similar relaxation behavior to the same 
equilibrium value for different sets of the initial conditions. We thus conclude numerically that the 
system possesses the mixing property. Therefore, the ensemble of the initial conditions at t = is 
regarded as the microcanonical ensemble with the energy Eq when the ensemble is made by sufficiently 
long time evolution of phase space points sampled from a distribution absolutely continuous to the 
Lebesgue measure on the energy surface. Here, we remark that the relaxation curve includes an 
oscillatory component, while the envelop curve exhibits an exponential decreasing behavior. Both 
the period of the oscillation and the relaxation time seem to be larger for larger A^. 



(a) 



0.03 



< 0.025 




200 




200 



Fig. 1. Relaxation behavior of the ensemble average of A. A'^ = 20. The ensemble of initial conditions is made artificially 
with fixing E and A as E = 1 and ^4(0) — 0.01. (b)log | (^4) — ^cq| versus t. Acq is the equilibrium value determined 
by the graph (a). The dotted line shows | (A) - Aoq| = exp(-t/50)/100. 



In order to demonstrate the chaotic nature quantitatively, we show the Lyapunov exponents in 
Fig. ^. Notice that there are two additional zero Lyapunov exponents because of the momentum 
conservation. That is, A'^p = N — 2. The convergence of orthonormal frames is confirmed in the 
way described in Section 4.2. Figure |3| shows the time evolution of the average of the distance 
d{J^{t, r{0))ei, J^{t, r{0))e'j^) over initial conditions, where i = 1,2, N — 3 and N — 2. We see that the 
distance decreases to a computational noise level after t = 3000. 
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i 

Fig. 2. Lyapunov spectrum. A'' — 20. 




Fig. 3. Convergence of a set of orthogonal unit vectors. A'" = 20. 

8.3. Boltzmann entropy 

The Boltzmann entropy S-Q{E,g) is calculated numerically in the following way: First, according 
to the adiabatic theorem, which was discussed in Section 3.1, the phase space volume enclosed by an 
energy surface is conserved along quasi-static processes. Therefore, the equality 

Q{E,g) = Q{E^,{)) (8-4) 

holds for the quasi-static process {E, g) (£'*, 0). Since the FPU model with (7 = is reduced to the 
harmonic oscillator model, i7(ii^*, 0) is given by the volume of the — 2 dimensional sphere, and is 
calculated as 

n{E,,Qi) = cE^-\ (8-5) 
where c does not depend on E^,. Thus, the Boltzmann entropy at {E,g) can be evaluated aM 

SBiE,g) = SB{E,,0) = (iV-l)log£„ (8-6) 

where an additive constant with respect to E^ is omitted. 

In numerical experiments, a quasi-static process {EQ,go) ^ {Ei,gQ + Ag) is realized by the large 
r limit of the protocol g{t) = go + Agt/r for < t < r. In Fig. 4, the average and deviation of Ei 
are plotted against r. We find that the deviation becomes smaller for larger r, and we may assume 

* In our previous paperB, the factor in the right-hand side was written as A'' — 2, not TV — 1. This was a mistake. 
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that quasi-static processes are realized when r > 100. The equi-entropy curve through = {Eq^qq) 
in Fig. ^ was obtained in this way. We express the curve hy E = Eq^id', ^o)- Similarly, as shown in 
Fig. |5|, we can draw equi-entropy curves in the {E,g) space. 



(a) 

uT 



0.974 



0.973 




100 




100 



Fig. 4. (a) Average value of final energy versus r. (b) Deviation of values of final energy versus r. A*' = 20 and 
Ag = -1.0. 




10 20 30 

g 

Fig. 5. Equi-entropy curve through {Eo,go) (solid curve) and other equi-entropy curves (dotted line). 

Now, we consider a step process realized by the instantaneous change of the value of g, from gQ 
to gi = go + ^9, at i = 0. Then, the energy after the switching becomes Ei, whose value depends on 
the choice of the initial condition. The entropy difference ASb is calculated by 

ASb = 5B(i?i,5i) - SB{Eo,go), (8-7) 

with the formula Eq. ( |8-6D . In Fig. |6|, the average of the entropy difference over the initial conditions, 
{ASb)q, is plotted against Ag. This graph shows that {ASb)q is positive. Also, as shown in Fig. 0, 
the relative fluctuation of ASb becomes less as is increased. This implies the existence of the large 
deviation property of ASb- 

8.4. Excess information loss 

We show the result of numerical calculation of the excess information loss for the step processes 
(£'o,go) {El, go + Ag). In Fig. ^, Hex{t, r{0)) for four choices of the initial condition r{0) are 
plotted against t. One can see that He^{oo, r{0)) is clearly defined. In Fig. we show the average 
of Hg^{t, r{0)) over the initial conditions chosen from the microcanonical ensemble on the energy 
surface Sq. {Hcx)q is given as the value at t = oo in this graph. 

In Fig. 10, {Hcx)q is plotted against Ag. From this graph, we can evaluate the value of ^{2Jq) 



Thermodynamic irreversibility from Hamiltonian Chaos 



37 



(a) 



(b) 



m 

< 



0.2 



0.1 




n r 

N=100 



N=20 • 
N=5 □ y^*- 




— 0.2 



CM 



m 
CO 

<l 



0.1 




-10 10 20 30 -10 10 20 30 

Ag Ag 

Fig. 6. Entropy difference versus Ag. The different normalizations for the A'' dependence are used in (a) and (b). 
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Fig. 7. The probability of the entropy difference. Ag = 10. 
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Fig. 8. Hax{t, r{0)) versus t for four different initial conditions. A*" = 20 and Ag — 20. 



by the equality 

(ifex)o = H^o)^g + o{Ag) (8-8) 

for Ag ^0. In a similar way, in principle, we can calculate the value of ^{^) at each energy surface. 
In particular, i^cx:qs at the step process is given by 

^^ex:qs = ^('^(^o) + ^{^l))^9, (8-9) 

where Ui = {Ei,gi) and recall Eq. ( [7- 121) for the definition of i^exiqs- 
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Fig. 9. Average value of Hc^(t, r(0)) versus t. N = 20 and Ag = 20. 
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Fig. 10. (-ffox)o versus Ag. Square and filled circle symbols represent the data for N — 5 and A'^ = 20, respectively. 
The different normalizations for the A'' dependence are used in (a) and (b). 

8.5. Main experiment 

In order to check the vahdity of Eq. ( 7-19| ), we need to calculate (i/ex:qs)o! average of -ffex:qs 



over the initial conditions. This is written as 



ex:qs/o 



(8-10) 



However, this average cannot be calculated efficiently, because Ui depends on the choice of the initial 
condition, and ^^(Z'l) is evaluated from a graph indicating Ag versus the average of ffex over initial 
conditions sampled from the micro canonical ensemble on Ui. We thus find out a way to avoid the 
calculation of -ffcx:qs- We perform the reversed experiments in which the parameter g is changed from 
gi to go with the initial state U'q = {Eqs{gi; UQ),gi). Suppose that the process U'q S'l is realized, 
where U'l depends on the choice of the initial condition on the energy surface U'q. Then, we can 
calculate the Boltzmann entropy change ASb and the excess information loss, H'^^, for the process 
U[. These satisfy the relation 



^0 



i {ASbYo = Wo - 



Ml) 



where ()q denotes the average over the initial conditions sampled from the microcanonical ensemble 
on the energy surface S'q, and i?ex:qs denotes the quasi-static excess information loss for the step 
process Sq ^ U[. Here, we can prove the estimation 



cx:qs/o 



+ H' 



cx:qs 



oiiAgf 



•12) 
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(proof) 



From the definition of i7ex;qs, we have 



H' 



cx;qs 



1 



'0 2 

Using this equation and Eq. ( |8-9| ), we obtain 



(-f^cx:qs)o + \-^ex:qs 







--{<P{S',) + {<!>{E[%)Ag. 

Recalhng the energy change for step processes, we expect 

S'^-^i^ 0[{Agf), 
So-^[r. 0{{Agf ). 

Substitution these estimation to Eq. (8-15) leads to Eq. ( ^-121) . 



Finally, from Eqs. ( [7-19D , ( 8-ll| ) and ( p-12 ), we obtain the equality 



-{{ASj,)^ + (Z\5b>;) = (i^cx)o + (^cx>'o + o{N, {Agf). 



(8-13) 

(8-14) 
(8-15) 

(8-16) 
(8-17) 

(q.e.d. 

(8-18) 



This relation can be checked numerically. As shown in Fig. 11, Eq. ( ^-181 ) seems valid, and therefore, 
our theoretical arguments turn out to be consistent. 



— 0.2 



X 

V 0) 



X 
(U 



0.1 - 




^ 0.2 0.4 
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Fig. 11. {-H'ox)o + (^i'cx)o versus (Z\5'b)q + {ASsYq- Square and filled circle symbols represent the data for iV = 5 and 
iV = 20, respectively. The sold line shows (//cx>o + (-ffDo = l/2[(ziSB>o + {ASuYo]. 



§9. concluding remark 

The essence of thermodynamic irreversibility is described by the entropy principle. Therefore, 
when one discusses thermodynamic irreversibility from dynamical systems, the purpose is to find a 
state variable satisfying the entropy principle. Our arguments stand on this natural viewpoint. In 
this paper, we have found that the irreversible information loss leads to the state variable which 
satisfies the entropy principle. 

We expect that our theory may be extended so as to apply to other dynamical systems without 
Hamiltonian. For example, in dissipative systems driven by external forces, a steady state is realized. 
The fluctuation properties have been discussed extensively. However, an attempt of the construction 
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of state theory seems scarcely. Although Oono and Paniconi have proposed an operational way to 
construct a state variable in steady state thermodynamics, its validity is not confirmed. a* We will 
attempt to construct non-equilibrium thermodynamic functions from dynamical system models by 
studying fluctuation properties. 

Much variety of nonlinear dynamics have been known in the context of fluid systems, granular 
systems, chemical reaction systems, and biological systems. In these systems, the notion of entropy 
is not self-evident, and hardly connected to that in thermodynamics. Even in such systems, it may 
be important to characterize a state of the systems in terms of state functions representing a relation 
between states. In particular, one of the most serious questions in science may be the boundary of 
the living state. One may ask why we cannot restore the living state when a life is dead. This is 
not the problem of thermodynamic irreversibility, but there is a sort of irreversibility in a biological 
world. However, the question is too general to be discussed scientifically. We should argue more 
restricted phenomena related to this question. The determination of cell types in cell differentiation 
processes may be a good candidate. As already studied by Kaneko et.al.,!^ from the dynamical 
system viewpoint, a cell society may be modeled by chemical networks with the variable number of 
cells. In these studies, Kaneko et.al. have found that the rule of the determination of cell types 
emerges. We expect that the rule might be formalized by state space theory which shares common 
features with thermodynamics. As developed in our theory concerning thermodynamic irreversibility, 
we hope to find a quantity representing a sort of relation associated with biological irreversibility. 
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Appendix 

We summarize basic properties of the fc-dimensional volume element and /c-fold exterior product 
of vectors.© 

Consider a A:-dimensional surface in the n-dimensional Euclid space R". The surface can be 
decomposed into a set of sufficiently small pieces of /c-dimensional parallelaids. A /c-dimensional 
parallelaid including a point x G R" is identified to that in a tangent space at x G R". 

The /c-dimensional volume of the projection of the parallelaid to a space spanned by {cj^ , 1 < 
j < k} is denoted by uji^i^...i^dxi^ ■ ■ ■ dxi^, where dxi^ ■ ■ ■ dxi^ is a /c-dimensional volume measure in 
the space spanned by {e,^, 1 < j < k}. A set {oJi-^...i^} is called the /c-dimensional volume element. 

Let us write the /c-dimensional volume element in a coordinate-free form. We consider the par- 
allelaid B made by a set of vectors 1 < i < /c}. We then define the /c-fold exterior product 
6i A • • • A 6fe as a map from a /c-dimensional parallelaids to its /c-dimensional volume of the projection 
to B. Explicitly, for a /c-dimensional parallelaid made by a set of vectors {oj, 1 < ? < /c}, the action 
of the map 6i A • • • A 6^ is defined as 

6i A • • • A 6fc • (ai, • • • at) = det^, (A-1) 

where Gij = {bi,aj). We write 6i A • • • A 6^ as A^^ibi, when the order of the vectors is uniquely 
guessed. 

Then, we can find an exterior product A^^^Wj such that 

^h-ik = /^Li^i ■ (eii, • • • CiJ. (A-2) 
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Since the exterior product is uniquely determined, A;-dimensional volume element is identified with 
the A:-fold exterior product. (A set of vectors {tOi, 1 < i < k} is not uniquely determined.) 

The k-iold exterior product A^^^bj has two important properties, multi-linearity and skew- 
symmetry. The multi-linearity is the relation 

61 A • • • A {cibi + Cib'i) A • • • A 6fc = Q^i A • • • A 6i A • • • A 5fc (A-3) 

+ c'jbi A • • • A 6- A • • • A 6fc (A-4) 

for arbitrary i, where q and c[ are numbers, and the skew-symmetry is 

Ati bi^ = sgn(n, • ■■,ii) A^i bi, (A- 5) 

where sgn(ii, • • • , ik) takes a value 1 when the permutation (1, • • • , /c) — > (ii, • • • , i^) is generated by 
even number of exchanges, otherwise it takes a value -1. 
Using the two properties, we can prove 

k 

Ati Gijhj = detg Ati bi. (A- 6) 

(proof) 



^UY.Gijbj= Gij,bj,A---AGkj,bj, (A-7) 

= E Gij,---Gkj,bj,A--- Abj, (A-8) 
= J2 s&'^ih ■ • • jk)Gij^ • • • Gkj^bi A • • • A 6fc (A-9) 

= detg^i A ••• Aftfc. (A- 10) 

(q-e.d) 

The A:-dimensional volume of the parallelaid B is calculated as \/detS, where Bij = {bi,bj). We 
represent it by | A^^^ bi\. 
(proof) 

We can find a set of orthogonal unit vectors {uj, I < i < k} which generate the vector space 
spanned by 1 < i < k}. (One may construct {ui, 1 < i < k} hy employing the Gram-Schmidt 
procedure.) Then, the fc-dimensional volume of B is given by 

I A^iftil = I A^ifti • (ni,---,Ufc)|. (A-11) 

Since bi can be expanded in the form 

k 

b^ = T.G^3Uj, (A- 12) 

i=i 

we obtain 

A^i h = detg A^i Ui, (A- 13) 

where we have used Eq. ( [A-6D . Using the identify 

A-=iUi • (ni,---,nfc) = 1, (A-14) 



we can rewrite Eq. ( A-11 ) as 

I Ail bi\ = \detg\. (A-15) 
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On the other hand, since 



{bi, bj) = ^ GiiGjm{ui,Um), (A-16) 

Im 

= J2GuGji, (A-17) 



we have 

Therefore, the equahty 



(QQ^).., (A-18) 
B = ggl (A-19) 



det^ = {detgf (A-20) 
holds. Substituting this equahty into Eq. ( |A-15| ), we obtain 



I A*Li bi\ = VdetB. (A-21) 

(q.e.d) 

Further, for arbitrary I such that 1 < I < k, the inequahty 

1^1 A • • • A 6fe| < |6i A • • • A bi\\bi+i A • • • A 6fc| (A-22) 

holds. This makes us possible to define 'angle' (p between bi A • • • Abi and bi^i A • • • A 6^ in such a 
way that 

|5i A • • • A = |6i A • • • A bi\\bi+i A • • • A fefcl sin^. (A-23) 

(proof) 

As seen in the previous proof, there exist two sets of orthogonal unit vectors {uj, 1 < i < /} and 
{n^, I + 1 < i < k} such that 

bi A ■ ■ ■ A bi = \bi A ■ ■ ■ A bi\ui A ■ ■ ■ A ui, (A-24) 
6^+1 A • • • A 6fc = A • • • A bk\u'i^i A • • • A 4, (A-25) 

where notice that Ui is not orthogonal to u'j. Then, we have 

\biA---Abk\ = \bi A--- Abi\\bi+i A ••• A6fc||ni A ■ ■ ■ A ui A u'i_^^ A ••• a4|. (A-26) 

Now, by using the Gram-Schmidt orthogonalization, we define a new set of vectors {uj, / + 1 < 
j < k} as 



with 



Here, from the equality 



i-1 



Sj = \Uj - {uj,Um)Um\- (A-28) 



771=1 



i-1 j-i 

~ {u'j,Um)Um\'^ + I {u'^,Um)Um? = 1, (A-29) 



771=1 m=l 



we find 



< < 1. (A-30) 
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Then, from 



(A-31) 



m=l 




Sl+l • • • Sfc- 



(A-32) 



Substituting Eq. ( |A-32D into Eq. ([A-26| ), we finally obtain 



61 A ••• A6fc| = |6i A ••• A6i||6;+i A 
< \biA---Abi\\bi+iA 



A 6fc|s/+i • • • Sfc 
Abk\. 



(A-33) 
(A-34) 



(q.e.d) 
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